Abstract
We are concerned here with explaining how successful rule-following behavior might evolve and how an old evolved rule might come to be successfully used in a new context. Such rule-following behavior is illustrated in the transitive judgments of pinyon and scrub-jays (Bond et al., Anim Behav 65:479–487, 2003). We begin by considering how successful transitive rule-following behavior might evolve in the context of Skyrms–Lewis sender–receiver games (Lewis, Convention. Harvard University Press, Cambridge, 1969; Skyrms, Philos Sci 75:489–500, 2006). We then consider two ways that an agent might come to use an old evolved rule in a new context. The first involves the agent evolving successful dispositions for one concrete type of experience, then associating a new type of experience with the old evolved dispositions. The second involves the agent evolving dispositions that represent a general inferential schema, then composing these dispositions with others in a way that allows the agent to make inferences concerning a new concrete type of experience.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A rule is understood here as being implemented in the dispositions of an agent. In particular, rule-following behavior consists in the agent taking stimuli as input and producing an action as output. Rule-following in this sense, then, is easy. In contrast with the traditional Kripke-Wittgenstein rule-following worries, we are not concerned here with what the agents themselves may or may not know concerning the rules they are in fact following (Kripke 1982). Rather, we are concerned with how the agents might evolve successful rule-following behavior and how such successful rule-following might come to be extended to novel contexts. One might think of the present discussion, then, as providing a basic externalist evolutionary account of successful rule-following behavior.
Simple sender–receiver games were first used by David Lewis (Lewis 1969) to account for the formation of convention. Brian Skyrms later described how they might be formulated in an evolutionary context without any assumptions of ideal rationality or common knowledge (Skyrms 2006). The type of invention-learning dynamics we will use here was proposed by Skyrms (Skyrms 2010). It is an extension of the sort of simple reinforcement learning proposed by Roth and Erev (Roth and Erev 1995). Some of its formal properties are considered by Skyrms, Alexander, and Zabell (Alexander et al. 2011). The present paper is closely related to Brian Skyrms’ evolutionary models for how basic logical inference might evolve (Skyrms 2000) and my evolutionary models for how basic arithmetic language and practice might coevolve (Barrett 2013a). See also Simon Huttegger’s discussion of the evolution of meaning (Huttegger 2007).
The difference in accuracy is not statistically significant here, but there was a difference in the competence exhibited by the two species that depend on the position of stimuli in the overall implicit ordering. While the pinyon jay performance was equally good for all color pairs, one scrub-jay, for example, exhibited 92 % accuracy in responding to the color pair (1,2) but dropped to well below chance (37 %) for the color pair (4,5). Further, while the highest-ranked stimulus in the pair did not matter to the accuracy of the pinyon jays, they were significantly slower in responding to pairs that were lower in the sequence; and while the scrub-jays exhibited a clear first-item effect on accuracy, they only exhibited slight latency effects for stimuli that were lower in the sequence. The experimenters took this as evidence that there is a basic difference in the way each species represents the implicit color order [(Bond et al. 2003), p. 484].
The positional effect discussed in footnote 3 also suggests that the pinyon jays are better at getting a linear representation over all of the colors. This is part of the evidence that the different species use different representational strategies. See D'Amato and Columbo (1988), Terrace and McGonigle (1994), and Delius and Siemann (1998) for examples of alternative models for transitive behavior as exhibited by humans and other species.
While useful, the distinction between the agents’ first-order dispositions (their dispositions to signal and to act) and their second-order dispositions (the learning dynamics that updates their first-order dispositions) is at best a rough one. It begins to unravel if their second-order dispositions are themselves allowed to evolve in response to the evolution of their first-order dispositions. Indeed, we will allow for precisely this when we discuss how an agent’s second-order dispositions may evolve to fit old first-order dispositions to a new type of experience.
For discussions of standard Skyrms–Lewis sender–receiver games and other variants see Lewis (1969), Skyrms (2006), Argiento et al. (2009), Barrett (2007, 2009, 2013b), and Skyrms (2010). While we will restrict our attention here to evolution in the context of learning models, one should expect similar results in the context of population models as there is, for example, a close formal relationship between evolution under reinforcement learning and evolution under the replicator dynamics.
Note that here we are allowing for any pair of colors to be selected and even for a color to be paired with itself.
This dynamics describes a very simple sort of reinforcement learning with invention. Note that if a coder drew a black ball and if the action was successful, then the coder has invented a new type of signal he might send. The actor must then add urns corresponding to the new pairs of signals he may receive. Also, note that a new signal type is only kept if the first-play of the new type was successful, in which case, the new type becomes available for any representational purpose. As a result, the new signal type may well not evolve to mean what it meant on its first use.
On simulation, after 1 × 107 plays, the cumulative success rate is better than 0.75 about 99 % of the time; and, in general terms, the more plays, the better the cumulative success rate on simulation. If one only requires that the system evolve the distinction between a ≥ b and a < b, then after 1 × 107 plays the cumulative success rate is over 0.80 about 97 % of the time, which is about the accuracy of the pinyon jays and scrub-jays. If one changes the learning dynamics to allow for both reinforcement on success and punishment on failure, then the convergence rates on simulation are orders of magnitude faster. In this sense, the simple reinforcement learning dynamics we are considering here might be thought of as difficult evolutionary context. The thought is that evolutionary stories that can be told here can be expected to be relatively robust.
After 1 × 107 plays the cumulative success rate is over 0.90 about 85 % of the time.
For systems like those discussed here, this is typically a statistical rule that evolves to exhibit relatively stable behavior.
It is assumed in the setup of a sender–receiver game that the senders have second-order dispositions that allow them to individuate types of states of nature and send signals on the basis of these state types. How their second-order dispositions individuate states of nature ultimately plays a role in determining what sort of signaling system they evolve. In this sense there is always an implicit similarity relation at work in the individuation of states of nature in a sender–receiver game. States of nature that are in fact grouped together under this relation will end up being treated the same way by the evolved representational system. Here we are allowing the second-order dispositions that represent this similarity relation to evolve.
On simulation, the mean number of random switches that an agent must consider to get to the optimal pairing of tones and colors is about 40.82. Since choosing the most successful system is a matter of statistical inference, an agent following this learning dynamics faces a series of two-armed bandit problems. See Berry and Fristedt (1985) and, more recently, Zollman (2010) for discussions bandit problems and strategies for addressing them. In this particular case, the difference between the success rates of better and worse systems is relatively large, so while the agent may sometimes choose a worse system as better, it will not take much investigation to make the chance of doing so very small. More generally, however, the cost of comparing the success of alternative systems could make this learning dynamics less efficient.
Such analogical behavior explains the differential behavior of the pinyon and scrub jays. In particular, Bond, Kamil, and Balda conclude that the social ordering that pinyon jays must navigate within a large stable flock explains their ability to get the full linear ordering right while scrub-jays, who typically live in pairs, are more likely to get later parts of the ordering wrong. More generally, they argue that animals living in large social groups should exhibit an enhanced capacity for transitive inference on partial evidence because they already require such inferential capacities for successful social interactions (Bond et al. 2003, pp. 479, 484–485).
Since both evolve in the same general way, the distinction between basic and higher-order representational systems has more to do with what sort of pattern the systems end up representing and how the systems get used. While a basic system may evolve to represent a particular ordering among colors and take colors as inputs, a higher-order system may evolve to represent a property of an ordering generally and take actions of other representational systems as inputs.
It does not matter what these input types are as long as (1) they in fact exhibit the general pattern of a linear order as described here and (2) the action of a basic color-ordering system produces a state that the higher-order system counts as one of these input types. This second condition allows the actions of a basic representational system to serve as a type of signal to a higher-order system. Note, however, that while the higher-order system must be able to take input from the actions of other systems, the significance of this input is not prearranged. Rather, in order for an agent to be successful, the coordination of the outputs of one system with the inputs of the next is something that one would expect to evolve as the result of the relative success of different ways of composing simpler representational systems to form more complex systems. We will discuss this further below.
On simulation, after 1 × 107 plays, the cumulative success rate is better than 0.95 about 96 % of the time.
Note that the condition b l = a r can be reliably judged by agents who have evolved a basic system for adjacent colors as described above since they have also been trained up on identical colors.
Note that the higher-order system would often be taking the results of its own actions as inputs on such a recursive application.
One might think of the actions produced by the basic systems as implemented by the jays as internal actions in thought. More specifically, one might imagine that the actions a l > b l and a r > b r of the basic color system produce brain records that are then read as input to the evolved higher-order inferential system. Of course, this recursive model may have precious little to do with actual pinyon jay cognition. Rather, it is just a how-possible explanation for the evolution of the sort of rule-following behaviors they exhibit.
Setting aside the a?c action, there are, on a relatively liberal notion of how one might compose subsystems and individuate functionally distinct composite systems, fourteen functionally distinct ways to arrange the basic and higher-order subsystems and six ways to associate the outputs of each of the two earlier subsystems with the inputs of a later system in the chain, for a total of 504 functionally distinct ways to compose the subsystems. In addition, there are fourteen functionally distinct ways to impose identity constraints on the initial inputs to the composite system. There are then 7,056 functionally distinct constructions and identity constraints to be explored by an evolutionary process.
Barrett (2013b) for an extended discussion of what aspects of nature matter for success in the sense of success appropriate to such evolutionary games.
References
Alexander JM, Skyrms B, Zabell S (2011) Inventing new signals. Dyn Games Appl 2:129–145
Argiento R, Pemantle R, Skyrms B, Volkov S (2009) Learning to signal: analysis of a micro-level reinforcement model. Stoch Proc Appl 119:373–390
Barrett JA (2007) Dynamic partitioning and the conventionality of kinds. Philos Sci 74:527–546
Barrett JA (2009) Faithful description and the incommensurability of evolved languages. Philos Stud 147(1):123–137
Barrett JA (2013a) On the invention of numbers and the nature of mathematical knowledge. (forthcoming)
Barrett JA (2013b) On the coevolution of theory and language and the nature of successful inquiry. Erkenntnis. doi:10.1007/s10670-013-9466-z
Berry DA, Fristedt B (1985) Bandit problems: sequential allocation of experiments. Chapman and Hall, London
Bond AB, Kamil AC, Balda RP (2003) Social complexity and transitive inference in corvids. Anim Behav 65:479–487
D'Amato MR, Columbo M (1988) Representation of serial order in monkeys (Cebus apella). J Exp Psychol Anim Behav Process 14:131–139
Delius JD, Siemann M (1998) Transitive responding in animals and humans: exaptation rather than adaptation? Behav Process 42:107–137
Huttegger S (2007) Evolution and the explanation of meaning. Philos Sci 74:1–27
Kripke S (1982) Wittgenstein on rules and private language: an elementary exposition. Harvard University Press, Cambridge, MA
Lewis D (1969) Convention. Harvard University Press, Cambridge, MA
Roth Al, Erev I (1995) Learning in extensive form games: experimental data and simple dynamical models in the intermediate term. Games Econ Behav 8:164–212
Skyrms B (2000) Evolution of Inference. In: Tim K, George G (eds) Dynamics of human and primate societies. Oxford University Press, New York, pp 77–88
Skyrms B (2006) Signals. Philos Sci 75:489–500
Skyrms B (2010) Signals evolution, learning, & information. Oxford University Press, New York
Terrace HS, McGonigle B (1994) Memory and representation of serial order by children, monkeys, and pigeons. Curr Dir Psychol Sci 3:180–185
Zollman K (2010) The epistemic benefit of transient diversityy. Erkenntnis 72:17–35
Acknowledgments
I would like to thank Brian Skyrms, Simon Huttegger, and Jason Alexander for helpful discussions and Martha Barrett for helpful comments on an earlier draft. This paper was written while visiting the Zukunftskolleg, Universität Konstanz. In this regard, I would also like to thank Franz Huber.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Barrett, J.A. The Evolution of Simple Rule-Following. Biol Theory 8, 142–150 (2013). https://doi.org/10.1007/s13752-013-0104-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13752-013-0104-4