Hoffmann, Matej and Müller, Vincent C. (2014), 'Trade-offs in exploiting body morphology for control: From simple bodies and model-based control to complex ones with model-free distributed control schemes', in Helmut Hauser; Rudolf M Füchslin and Rolf Pfeifer (eds.), Opinions and Outlooks on Morphological Computation (Zurich: E-Book), 185-94. – http://www.sophia.de http://orcid.org/0000-0002-4144-4957 ISBN 978-3-033-04515-

Copyright This e-book has been realized with LaTeX and LyX. All photos used to produce the title pages of the individual chapters including the title page of this e-book have been provided by Rachel Amstrong. They have been adapted by Helmut Hauser with Pixelmator. All other figures used in the individual chapters have been provided by the authors. Figures taken from previous publications have been marked accordingly. This e-book is published under the license CC BY SA 4.0. This means you can and should share this e-book. However, you have to give appropriate credit and if your remix, transform or build upon this material you must distribute your contribution under the same license. Please note that figures previously published somewhere else (marked as such) are excluded from this license and you need the permission of the corresponding author/journal to reproduce them. 9 783033 045156 ISBN 978-3-033-04515-6 Introduction Morphological Computation is based on the observation that biological systems seem to carry out relevant computations with their morphology (physical body) in order to successfully interact with their environments. This can be observed in a whole range of systems and at many different scales. It has been studied in animals – e.g., while running, the functionality of coping with impact and slight unevenness in the ground is "delivered" by the shape of the legs and the damped elasticity of the muscle-tendon system – and plants, but it has also been observed at the cellular and even at the molecular level – as seen, for example, in spontaneous self-assembly. The concept of morphological computation has served as an inspirational resource to build bio-inspired robots, design novel approaches for support systems in health care, implement computation with natural systems, but also in art and architecture. As a consequence, the field is highly interdisciplinary, which is also nicely reflected in the wide range of authors that are featured in this e-book. We have contributions from robotics, mechanical engineering, health, architecture, biology, philosophy, and others. On one hand, the beautiful richness of this interdisciplinarity has a lot of potential, but on the other, it poses a number of serious challenges. For example, people with different backgrounds have a different understanding of the concepts involved. They use different terms for the same ideas or, even more problematic, they use the same terms to mean something often substantially different. Also, the tools and methods are frequently quite apart, as illustrated, e.g., by engineering and neuroscience. Nevertheless, we strongly believe that this interdisciplinarity is a big opportunity, which enables the field to grow and to cross-fertilize. Scientists from other areas can provide novel points of view, they can challenge long standing, many times implicit and unquestioned assumptions inherent to a field, and they can help to find disruptive approaches to solve notorious problems. We would even argue that a lot of times truly novel and seminal work happens exactly at the intersection of different fields. Based on the strong belief that – if done right – interdisciplinary research can be enormously productive we had initiated this e-book. The intention was to collect opinions on morphological computation from researchers from a broad range of research as well cultural backgrounds. Instead of the formal style of publications of scientific results, we wanted to provide a rather informal environment where people can present their point of view on morphological computation, its future trajectory and its possibly far-reaching implications. The book should be an inspirational resource and it should allow for cross-fertilization between different disciplines. One of our goals was to make sure that the contributions are written in a style accessible for a broad audience. Moreover, it was clear from the very beginning that this e-book should be available for free to lower the threshold to read it. In our opinion we have been able to meet the objectives of the e-book, especially, because of the help of all authors who took the effort to write excellent contributions. Introduction We are truly grateful for all their work and we also appreciate their patience to "hang in there" with us so long. Finally, we hope that this colorful collection of thoughts will inspire researchers, engineers, students, teachers, and anyone with an interest in scientific debate, alike and will generate a lot of discussion and ultimately novel scientific topics. The editors: Helmut Hauser, Rudolf M. Füchslin, and Rolf Pfeifer 4 Acknowledgement The editors Helmut Hauser, Rudolf M. Füchslin, and Rolf Pfeifer would like to thank all contributors for their exciting and inspiring submissions. We also would like to thank for their patience and that, despite the long time it took to realize this e-book, they never lost their believe in this projects. In addition, we would like thank Rachel Armstrong for her great photos that have been used for all title pages. List of Contributions 1. "Extracting the Full Power of Morphological Computation: Lessons from Case Studies of Robots under Decentralized Control" by Akio Ishiguro [link] 2. "A Reservoir Computing View of Morphological Computation" by Ken Caluwaerts and Benjamin Schrauwen [link] 3. "Deep into Morphology: Emotions and Functional Structure" by Carlos HerreraPérez and Ricardo Sanz [link] 4. "Zen, Robotics and the Art of Pushing Swings" by Fabio Bonsignorio [link] 5. "A Virtual Material Approach to Morphological Computation" by Jeff D. Jones [link] 6. "A Review of Morphological Computation from a Perspective of Heterarchy" by Kazuto Sasai [link] 7. "Morphological Computation and Heuristic Bio-Robotics" by Koh Hosoda [link] 8. "Morphology: A Concrete Form of Intelligence" by Murat Reis and Cihat Ensarioglu [link] 9. "Molecules and Robots" by Shuhei Miyashita [link] 10. "Morphological Computation: A Perspective Based on Bacterial Movement" by Surya G. Nurzaman, Yoshio Matsumoto, Yutaka Nakamura, Kazumichi Shirai, Satoshi Koizumi, Fumiya Iida, and Hiroshi Ishiguro [link] 11. "Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators" by Takuya Umedachi and Akio Ishiguro [link] 12. "Evolving Morphological Computation" by Josh Bongard [link] 13. "A Morphological Viewpoint: Juxtaposition of Design Approaches for LocomotionRehabilitation Robotics" by Kenneth J. Hunt and Juan Fang [link] 14. "Morphological Computing and Design" by Rachel Armstrong [link] 15. "Morphological Computation at the Molecular Scale" by Hamada Shogo [link] 16. "Morphological Computation A Broad Perspective" by Wolfgang Banzhaf [link] 17. "Trade-Offs in Exploiting Body Morphology for Control: from Simple Bodies and Model-Based Control to Complex Bodies with Model-Free Distributed Control Schemes" by Matej Hoffmann and Vincent C. Müller [link] List of Contributions 18. "Morphological Control as Guiding Principle in Physiology and Medical Applications" by Rudolf M. Füchslin, Helmut Hauser, Irene Poli, Roberto Serra, Marco Vilani, Stephan Scheidegger, and Mathias S. Weyland [link] 19. "The Morphological Computation Principles as a New Paradigm for Robotic Design" by Davide Zambrano, Matteo Cianchetti, and Cecilia Laschi [link] 20. "Morphological Computation The Body as a Computational Resource" by Helmut Hauser, Kohei Nakajima, and Rudolf M. Füchslin [link] 7 How to cite this e-Book The entire e-book can be cited as following. Please add also a URL to a copy of the book to facilitate sharing: @book {e-book_on_morphological_computation_2014, title = {Opinions and Outlooks on Morphological Computation}, booktitle = {Opinions and Outlooks on Morphological Computation}, editor = {Hauser, Helmut and Füchslin, Rudolf M. and Pfeifer, Rolf}, isbn = {ISBN 978-3-033-04515-6}, year = {2014}, url = {LINK_TO_YOUR_COPY} } The individual chapters can be cited as following by changes bold entries to your corresponding information: @incollection {BIB_REF_NAME_2014, title = {YOUR_CHAPTER_TITLE}, author = {LIST_OF_AUTHORS}, booktitle = {E-book on Opinions and Outlooks on Morphological Computation}, editor = {Hauser, Helmut and F\"{u}chslin, Rudolf M. and Pfeifer, Rolf}, pages = {FROM–TO} chapter = {CHAPTER_NR}, isbn = {978-3-033-04515-6} year = {2014} url={LINK_TO_YOUR_ONLINE_COPY} } List of Authors • Armstrong, Rachel – [email] – Experimental Architecture, University of Newcastle, Department of Architecture, Planning and Landscape, UK • Banzhaf, Wolfgang – [email] – Department of Computer Science, Memorial University of Newfoundland, Canada • Bongard, Josh – [email] – Morphology, Evolution and Cognition Laboratory, Department of Computer Science, University of Vermont, USA • Bonsignorio, Fabio – [email] – Institute of Biorobotics Scuola Superiore S. Anna and HeronRobots srl, Italy • Caluwaerts, Ken – [email] – Reservoir Lab, Electronics and Information Systems Department, Ghent University, Belgium • Cianchetti, Matteo – [email] – The BioRobotics Institute of the Scuola Superiore Sant'Anna, Pisa, Italy • Ensarioglu, Cihat – [email] – Mechanical Engineering Department, Uludag University, Bursa, Turkey • Fang, Juan – Institute for Rehabilitation and Performance Technology, Bern University of Applied Sciences, Switzerland • Füchslin, Rudolf M. – [email] – Applied Complex Systems Science Group at Zurich University of Applied Sciences, Switzerland, and European Centre for Living Technology (ECLT) in Venice, Italy • Hamada, Shogo – [email] – Department of Bioengineering and Robotics, Graduate School of Engineering, Tohoku University, Japan • Hauser, Helmut – [email] – Artificial Intelligence Laboratory, University of Zurich, Switzerland • Herrera-Pérez, Carlos – [email] – Autonomous Systems Laboratory, Universidad Politecnica de Madrid, Spain • Hoffmann, Matej – [email] – iCub Facility, Istituto Italiano di Tecnologia, Genoa, Italy • Hosoda, Koh – [email] – Department of Systems Innovation, Graduate School of Engineering Science, Osaka University, Japan List of Authors • Hunt, Kenneth J. – [email] – Institute for Rehabilitation and Performance Technology, Bern University of Applied Sciences, Switzerland • Iida, Fumiya – [email] – Head of the Bio-Inspired Robotics Laboratory, ETH Zürich, Switzerland • Ishiguro, Akio – [email] – Professor at the Research Institute of Electrical Communication, Tohoku University, Japan • Ishiguro, Hiroshi – Director of the Intelligent Robotics Laboratory, Department of Systems Innovation, Osaka University, Japan • Jones, Jeff Dale – [email] – Research Fellow at the Centre for Unconventional Computing, University of the West of England, UK • Koizumi, Satoshi – Intelligent Robotics Laboratory, Department of Systems Innovation, Osaka University, Japan • Laschi, Cecilia – [email] – The BioRobotics Institute of the Scuola Superiore Sant'Anna, Pisa, Italy • Matsumoto, Yoshio – [email] – Service Robotics Laboratory, AIST, Japan • Miyashita, Shuhei – [email] – Computer Science and Artificial Intelligence Lab, Boston MIT, USA • Müller, Vincent C. – [email] – Division of Humanities & Social Sciences, Anatolia College/ACT, Pylaia-Thessaloniki, Greece and University of Oxford, UK • Nakajima, Kohei – [email] – The Hakubi Center for Advanced Research and Nonequilibrium Physics and Theoretical Neuroscience, Department of Applied Analysis and Complex Dynamical Systems, Graduate School of Informatics, Kyoto University, Japan • Nakamura, Yutaka – [email] – Intelligent Robotics Laboratory, Department of Systems Innovation, Osaka University, Japan • Nurzaman, Surya G. – [email] – Bio-Inspired Robotics Laboratory, ETH Zürich, Switzerland • Poli, Irene – [email] – European Centre for Living Technology, Department of Statistics, University Ca'Foscari of Venice, Italy • Reis, Murat – [email] – Mechanical Engineering Department, Uludag University, Bursa, Turkey • Sanz, Ricardo – [email] – Autonomous Systems Laboratory, Universidad Politecnica de Madrid, Spain 10 List of Authors • Sasai, Kazuto – [email] – Research Institute of Electrical Communication, Tohoku University, Japan • Scheidegger, Stephan – [email] – Institute of Applied Mathematics and Physics, Zurich Univ. of Applied Sciences, Switzerland • Schrauwen, Benjamin – [email] – Reservoir Lab, Electronics and Information Systems Department, Ghent University, Belgium • Serra, Roberto – [email] – European Centre for Living Technology in Venice and Department of Communication and Economics, University of Modena and Reggio Emilia, Italy • Shirai, Kazumichi – Intelligent Robotics Laboratory, Osaka University, Japan • Umedachi, Takuya – [email] – Department of Biology, Tufts University, USA • Vilani, Marco – European Centre for Living Technology in Venice and Department of Communication and Economics, University of Modena and Reggio Emilia, Italy • Weyland, Mathias S. – [email] – European Centre for Living Technology, Venice, Italy • Zambrano, Davide – [email] – The BioRobotics Institute of the Scuola Superiore Sant'Anna, Pisa, Italy 11 Abstract: The concept of morphological computation could be the key to understanding animals' adaptive behavior. Despite its appeal, its mechanism and application method remain elusive. In this article, we consider how morphological computation can be used effectively by taking some of our studies as practical examples. Through these case studies, we show that the concept of morphological computation can be exploited effectively when we integrate it into autonomous decentralized control systems. We also discuss issues requiring further consideration for the effective use of morphological computation. Extracting the Full Power of Morphological Computation Introduction Animals, even primitive living organisms, do not lose their functionality under unstructured, unpredictable real-world constraints, and they adapt to various environments in real time, even though their computational resources are limited. This ability has been honed by evolutionary selection pressure, and it is likely that there is an ingenious underlying mechanism essentially different from the control mechanisms used in conventional robots. Clarifying this remarkable mechanism will bridge the gap between robots and animals. Now, a question arises: what are the mechanisms underlying animals' adaptive behavior? The concept of morphological computation could be the key to answering this question [2, 34, 35]. Animal locomotion is not generated merely from a control system, but rather from tight interaction between a control system, a mechanical system, and the real-world environment. This automatically suggests the following conclusion: a certain amount of computation should be off-loaded from the control system to the mechanical system. A beautiful well-known example of robots successfully using morphological computation is (pseudo) passive dynamic walkers [25, 3]. However, despite their highly energy efficient walking, the environments to which they are able to adapt were greatly limited. This strongly suggests that the way of applying morphological computation still leaves room for discussion. Because the concept of morphological computation is still in its infancy, it is of great worth to accumulate various case studies. Considering the above, we have been investigating how morphological computation can be used effectively through various robotic case studies. In this article, we show that it can be exploited effectively when we integrate it into autonomous decentralized control systems, in which the coordination of simple individual components yields non-trivial macroscopic behavior or functionalities, by considering some of our representative case studies. Robotic Case Studies In the following, we introduce three robotic case studies conducted by our research group (i.e., Slimebot, Slimy, and Oscillex), each of which investigates the extensive use of morphological computation in the context of autonomous decentralized control. Slimebot In the first case study, we consider a two-dimensional modular robot called Slimebot. Modular robots – also referred to as reconfigurable robots – have been attracting considerable attention [7, 26]. Because the relative positional relationship between the modules can be altered according to the situation encountered, a modular robot is expected to show significant abilities (e.g., adaptability, scalability) that are unavailable in a robot having a fixed morphology. Despite such potential abilities, we must note that in most of the modular robots developed so far, the modules have typically been connected mechanically and/or elec13 Extracting the Full Power of Morphological Computation tromagnetically by highly rigid mechanisms. Under this type of rigorous connectivity control mechanism, however, the required control algorithm may be extremely complicated and intractable because it has to always specify which modules should be connected physically as well as how each module should be moved. In addition, module connections implemented by such a highly rigid mechanism may reduce some of the advantages that can be expected, particularly the flexibility against environmental changes. To overcome these drawbacks, we designed an unconventional decentralized control scheme for modular robots inspired by a true slime mold (Physarum polycephalum), which is a primitive living organism whose behavior is generated by a purely decentralized control mechanism based on coupled biochemical oscillators similar to a central pattern generator (CPG)[40, 39]. A significant feature of Slimebot is that we explicitly exploit emergent phenomena arising from the interplay between the control and mechanical systems in order to control the morphology in real time. To this end, we focus on a functional material and mutual entrainment; the former is used as a spontaneous connectivity control mechanism between the modules, and the latter acts as the core of the control mechanism for the generation of locomotion and ensures the scalability and coherency. Note that the spontaneous connectivity control mechanism achieved by the functional materials allows Slimebot to exploit morphological computation effectively: Slimebot can deform in ways favorable to the motion underway. Figure 1 shows representative data on the morphological transitions. Interestingly, the method of negotiating the environment seems to vary significantly: in Figure 1(a), Slimebot passes through obstacles by narrowing the width of its entire system, whereas in Figure 1(b), it negotiates its environment by enclosing obstacles. Note that these behaviors are not preprogrammed but are totally emergent. The obtained results are expected to shed light on how the control and mechanical systems should be coupled, and what well-implemented morphological computation contributes to the resulting behavior. For the details on Slimebot, please refer to [20]. Slimy In the second case study, we also consider a soft-bodied amoeboid robot under decentralized control. Although we again focused on a true slime mold as the source of inspiration, this robot, Slimy, exploits morphological computation enabled by the softness of the body in a totally different manner from Slimebot. In the following, we will briefly explain the idea. One of the pivotal issues in designing an autonomous decentralized control system is the design of the local sensory feedback provided to an individual component. However, a logic connecting the behavior of an individual component to the behavior of the entire system that induces non-trivial macroscopic functionalities (e.g., adaptability, scalability, fault tolerance) has not yet been established. Therefore, it is undeniable that no consistent scheme for designing local sensory feedback mechanisms exists. To address this lack, we investigated a design scheme for effectively connecting the local behavior to the global behavior by constructing Slimy. It has three significant features: (1) Slimy has a truly soft, deformable body resulting from real-time tunable 14 Extracting the Full Power of Morphological Computation (a) With 100 modules (b) With 500 modules Figure 1: Representative data on the morphological transitions of Slimebot (left to right in each panel). Thick circles represent obstacles. The attractant acts from the top of the figure. Note that the number of time steps of the simulation for 100 and 500 modules are different due to the size of body. Figures taken from [20]. 15 Extracting the Full Power of Morphological Computation Figure 2: Representative data on the locomotion of Slimy through a tunnel-like narrow aisle. Figure taken from [42]. springs and protoplasm; the former are used for the body's outer skin, and the latter satisfies the law of conservation of mass; (2) a fully decentralized local sensory feedback mechanism is realized by exploiting the long-distance physical interaction between body parts arising from the law of conservation of protoplasmic mass, similar to that observed in waterbeds, which guarantees a connection between the local behavior and the global behavior; and (3) a systematic design scheme for the local sensory feedback mechanism based on a "discrepancy function"1 is introduced. Experimental results show that Slimy exhibits highly adaptive amoeboid locomotion without relying on any hierarchical structure. Figures 2 and 3 show representative data on Slimy's locomotion. Figure 2 shows simulation results and Figure 3 screenshots of the real robot. The obtained results are expected to shed new light on a design methodology for an autonomous decentralized control system as well as a way to exploit morphological computation so as to produce emergent non-trivial functionalities. For the details on Slimy, please refer to [42, 44, 23]. Oscillex In the third and final case study, let us look at how morphological computation can be exploited to induce adaptive legged locomotion. Quadrupeds are well known to have versatile gait patterns that depend on the locomotion speed, environmental conditions, and animal species [27, 18, 15, 16, 17, 1]. These locomotor patterns are generated via coordination between limb movements – interlimb coordination – and are partly controlled 1 In a nutshell, discrepancy function is a function that quantitatively measures the discrepancy taking place between the control system, mechanical system, and the environment. For detail, please see [43], for example. 16 Extracting the Full Power of Morphological Computation Figure 3: Screenshots of the locomotion of the real physical robot Slimy (view from top to bottom). The red line describes the trajectory of the center over time. Figure taken from [44]. 17 Extracting the Full Power of Morphological Computation by an intraspinal neural network called the CPG [12, 13]. Although this forms the basis for current control paradigms for interlimb coordination, the mechanism responsible for interlimb coordination remains elusive. Therefore, each individual CPG model proposed so far has been designed on a completely ad hoc basis by focusing on the interlimb neural connections in the CPG [38, 45, 21, 10, 11, 22, 41, 24, 36, 19]. To elucidate the ability of animals to generate adaptive interlimb coordination, we reworked the design principle of CPG-based control. Animal locomotion is not generated merely from neural systems, but rather from tight interaction between neural systems, musculoskeletal systems, and the real-world environment [38, 33, 34]. Thus, it is essential to elucidate the locomotion generation mechanism by analyzing the interaction dynamics between these three systems as well as by analyzing the neural systems themselves. Therefore, we hypothesize that interlimb coordination should rely more on morphological (or physical, i.e., non-neural) communication through body dynamics during leg movements rather than on explicit interlimb neural connections. On the basis of this consideration, we recently proposed an unconventional CPG model that consists of four decoupled oscillators with local sensory feedback from only a force sensor on each corresponding leg. Very briefly speaking, the key idea of our CPG model is that we designed local sensory feedback that uses the ground reaction force acting on each leg such that a leg remains in the stance phase while supporting the body. Despite its simplicity, our robot with this CPG model exhibits good adaptability to changes in weight distribution and walking speed, and it can mimic various walking patterns of actual quadrupeds (see Figures 4 and 5). This strongly suggests that physical interaction between the legs during movement is essential for interlimb coordination in quadruped walking. One plausible explanation for these results is that the proposed local sensory feedback system allows each leg to recognize the positional relationship of all the other legs, i.e., how the legs support the body at that particular moment, without having to perform computationally expensive calculations. This case study illustrates well the power of morphological computation for the generation of adaptive behavior. For the details on this CPG model, please refer to [31]. Discussion To simplify the following discussion, let us use Figure 6, which illustrates schematically the possible task distribution between neural computation (i.e., the control system) and morphological computation (i.e., the mechanical system). Note that the position of the slider handle indicates the degree to which neural computation and morphological computation contribute to the resulting behavior at a particular moment. Imagine we can freely blend these two types of computation by dragging the slider handle left or right. Now let us first move this slider handle all the way to the left. Note that this corresponds to a situation where the control system plays a dominant role in generating behavior. Most robots developed so far (e.g., walking robots controlled on a zero mo18 Extracting the Full Power of Morphological Computation Figure 4: Quadruped robot Oscillex. Each leg is controlled independently. Figure taken from [31]. Figure 5: Experimental results for the gait with changes in body properties. Two full gait cycles are shown for clarity. (A) With a load of 0.12 kg on the forelegs, and (B) with a load of 0.29 kg on the hind legs. The duty factor of the legs bearing the load (forelegs in (A) and hind legs in (B)) are larger than those of the legs without a load. Figure taken from [31]. ment point basis) have been controlled in this manner. Next, let us move the handle in the opposite direction, all the way to the right. In contrast to the previous case, in this situation the mechanical system has all the responsibility for generating behavior. Beautiful instantiation of this case are the passive dynamic walker or runner [25, 30, 28, 37, 29]. We will immediately notice in this figure that current robots are driven by these two 19 Extracting the Full Power of Morphological Computation Figure 6: Graphical representation of the possible task distribution between neural computation and mechanical computation and the resulting behavior. The position of the slider handle specifies how these two types of computation are coupled at a particular moment. Figure is from [20] and has been slightly revised. extremely different approaches, and very few exist in between. From this realization, we present the following conclusions: First, robots should be designed such that neural computation and morphological computation are blended in a well-balanced manner. Here the word "well-balanced" means that we do not intend this blending to be just a trade-off between neural computation and morphological computation; we expect that appropriate blending will induce various non-trivial functionalities that cannot be explained solely in terms of the control and mechanical systems themselves. Slimy, for example, exploits the long-distance physical interaction between body parts arising from the law of conservation of protoplasmic mass in morphological computation, which enables a sensory feedback mechanism using only locally available information that yields coherent global behavior without computationally expensive calculation. In addition, Slimy exhibits various non-trivial behaviors such as adaptability, fault tolerance, and scalability. Second, the way in which the two types of computation are blended (i.e., the position of the slider handle) should be varied in response to the situation. To discuss this point, it is well worth looking at how neurophysiologists have considered the roles of the CPG and sensory feedback in the generation of adaptive locomotion over the years. Grillner [12, 13] suggested the necessity of the CPG, whereas others argued the necessity of sensory feedback [4], which plays a crucial role in adapting motor patterns to the situation encountered, e.g., the locomotion speed and environmental conditions. More concretely, two sensory mechanisms have been considered important in controlling the stance-toswing transition of a walking cat: (1) unloading of the leg and (2) hip extension [14, 5, 32]. Furthermore, Ekeberg and Pearson [6] used a computer simulation to demonstrate that sensory information on the loading force from each leg plays a crucial role in the stanceto-swing transition. On the other hand, Full and Koditschek [8] and Ghigliazza et al. [9] have suggested that during rapid locomotion, mechanical feedback plays a more essential 20 Extracting the Full Power of Morphological Computation role than neural sensory feedback in generating locomotion. We expect Oscillex to illuminate this long-standing debate and provide a better understanding of how the autonomous blending of neural computation with morphological computation can be achieved effectively. The experimental results obtained with Oscillex indicate that the oscillator's regime shifts from the excitatory regime to the oscillatory regime with an increase in locomotion speed. This suggests that our CPG model can reproduce the autonomous transition from the domain of neural feedback (neural computation) in low-speed locomotion to that of mechanical feedback (morphological computation) in mediumor high-speed locomotion by exploiting the sensory feedback on the basis of physically reasonable sensory information on the loading force. For details on this topic, please refer to [31]. To realize a mechanism for autonomous blending of neural and morphological computation, it would also be interesting to consider an additional adaptation mechanism that governs, for example, the change in joint stiffness. This suggests that multi-time-scaled adaptation mechanisms should be implemented in robots. Our robotic case studies are a first step towards these points. Bibliography [1] M. Cartmill, P. Lemelin, and D. Schmitt. Support polygons and symmetrical gaits in mammals. Zool. J. Linn. Soc., 136:401–420, 2001. [2] P. Chandana. Morphological computation: A basis for the analysis of morphology and control requirements. Robotics and Autonomous Systems, 54(8):619–630, 2006. [3] S. Collins, A. Ruina, R. Tedrake, and M. Wisse. Efficient bipedal robots based on passive–dynamics walker. Science, 307:1082–1085, 2005. [4] J. Duysens and K. Pearson. The role of cutaneous afferents from the distal hindlimb in the regulation of the step cycle of thalamic cats. Exp. Brain Res., 24:245–255, 1976. [5] J. Duysens and K. Pearson. Inhibition of flexor burst generation by loading ankle extensor muscles in walking cats. Brain Res., 187:321–332, 1980. [6] O. Ekeberg and K. Pearson. Computer simulation of stepping in the hind legs of the cat: an examination of mechanisms regulating the stance-to-swing transition. J. Neurophysiol., 94:4256–4268, 2005. [7] T. Fukuda and T. Ueyama. Cellular robotics and micro robotic systems. World Scientific Series in Robotics and Automated Systems, 1994. [8] R. Full and D. Koditschek. Templates and anchors: Neuromechanical hypothesis of legged locomotion on land. J. of Exp. Biol., 202:3325–3332, 1999. 21 Extracting the Full Power of Morphological Computation [9] R. Ghigliazza, R. Altendorfer, P. Holmes, and D. Koditschek. Templates and anchors: Neuromechanical hypothesis of legged locomotion on land. J. of Exp. Biol., 202:3325–3332, 1999. [10] M. Golubitsky, I. Stewart, P. L. Buono, and J. Collins. A modular network for legged locomotion. Physica D, 115:56–72, 1998. [11] M. Golubitsky, I. Stewart, P. L. Buono, and J. Collins. Symmetry in locomotor central pattern generators and animal gaits. Nature, 401:693–695, 1999. [12] S. Grillner. Locomotion in vertebrates: central mechanisms and reflex interaction. Physiol. Rev., 55:247–304, 1975. [13] S. Grillner. Neurobiological bases of rhythmic motor acts in vertebrates. Science, 228:143–149, 1985. [14] S. Grillner and S. Rossignol. On the initiation of the swing phase of locomotion in chronic spinal cats. Brain Res., 146:269–277, 1978. [15] M. Hildebrand. Symmetrical gaits of horses. Science, 150:701–708, 1965. [16] M. Hildebrand. Symmetrical gaits of primates. Am. J. Phys. Anthropol., 26:119–130, 1967. [17] M. Hildebrand. Symmetrical gaits of dogs in relation to body build. Journal of Morphology, 124:353–360, 1968. [18] D. Hoyt and R. Taylor. Gait and the energetics of locomotion in horses. Nature, 292:239–240, 1981. [19] A. Ijspeert. Central pattern generators for locomotion control in animals and robots: A review. Neural Networks, pages 642–653, 2008. [20] A. Ishiguro, M. Shimizu, and T. Kawakatsu. A modular robot that exhibits amoebic locomotion. Robotics and Autonomous Systems, 54:641–650, 2006. [21] S. Ito, H. Yuasa, Z. W. Luo, M. Ito, and D. Yanagihara. A mathematical model of adaptive beheviour in quadruped locomotion. Biol. Cybern., 78:337–347, 1998. [22] H. Kimura, S. Akiyama, and K. Sakurama. Realization of dynamic walking and running of the quadruped using neural oscillator. Autonomous Robots, 7:247–258, 1999. [23] R. Kobayashi, A. Tero, and T. Nakagaki. Mathematical model for rhythmic protoplasmic movement in the true slime mold. Journal of Mathematical Biology, 53(1): 273–286, 2006. [24] C. Maufroy, H. Kimura, and Takase. Towards a general neural controller for quadrupedal locomotion. Neural Networks, 21:667–681, 2008. 22 Extracting the Full Power of Morphological Computation [25] T. McGeer. Passive dynamic walking. The International Journal of Robotics Research, 9(2):62–82, 1990. [26] S. Murata and H. Kurokawa. Self-organizing robots. Springer-Verlag, 2012. [27] R. Muybridge. Animal locomotion: The Muybridge work at the University of Pennsylvania. Ann Arbor, MI: University of Michigan Library., 1981. [28] K. Nakatani, Y. Sugimoto, and K. Osuka. Demonstration and analysis of quadrupedal passive dynamic walking. Advanced Robotics, 23(5):483–501, 2009. [29] D. Owaki, K. Osuka, and A. Ishiguro. Understanding the common principle underlying passive dynamic walking and running. Proc. of 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3208–3213, 2009. [30] D. Owaki, M. Koyama, S. Yamaguchi, S. Kubo, and A. Ishiguro. A 2D passive– dynamic–running biped with elastic elements. IEEE Transaction on Robotics, 27 (1):156–162, 2011. [31] D. Owaki, T. Kano, K. Nagasawa, A. Tero, and A. Ishiguro. Simple robot suggests physical interlimb communication is essential for quadruped walking. Journal of The Royal Society Interface, 2012. doi: 10.1098/rsif.2012.0669. [32] K. Pearson. Generating the walking gait: role of sensory feedback. Prog. Brain Res., 143:123–132, 2003. [33] R. Pfeifer and C. Scheier. Understanding intelligence. MIT Press, 1999. [34] R. Pfeifer, F. Iida, and G. Gomez. Morphological computation for adaptive behavior and cognition. International Congress Series, 1291:22–29, 2006. [35] J. A. Rieffel, F. J. Valero-Cuevas, and H. Lipson. Morphological communication: exploiting coupled dynamics in a complex mechanical structure to achieve locomotion. Journal of the Royal Society Interface, 7(45), 2010. doi: 10.1098/rsif.2009.0240. [36] L. Righetti and A. Ijspeert. Pattern generators with sensory feedback for the control of quadruped locomotion. Proc. of ICRA 2008, pages 819–824, 2008. [37] Y. Sugimoto, H. Yoshioka, and K. Osuka. Realization and motion analysis of multilegged passive dynamic walking. Proc. of SICE Annual Conference 2010, pages 2790–2793, 2010. [38] G. Taga, Y. Yamaguchi, and H. Shimizu. Self-organized control of bipedal locomotion by neural oscillators. Biol. Cybern., 65:147–159, 1991. [39] A. Takamatsu. Spontaneous switching among multiple spatio-temporal patterns in three-oscillator systems constructed with oscillatory cells of slime mold. Physica D: Nonlinear Phenomena, 223:180–188, 2006. 23 Extracting the Full Power of Morphological Computation [40] A. Takamatsu, R. Tanaka, H. Yamada, T. Nakagaki, T. Fujii, and I. Endo. Spatiotemporal symmetry in rings of coupled biological oscillators of physarum plasmodial slime mold. Physical Review Letters, 87:0781021, 2001. [41] K. Tsujita, K. Tsuchiya, and A. Onat. Decentralized autonomous control of a quadrupedal locomotion robot using oscillators. Artif. Life and Robotics, 5:152– 158, 2003. [42] T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. Fully decentralized control of a soft-bodied robot inspired by true slime mold. Biological Cybernetics, 102:261–269, 2010. [43] T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. Fully decentralized control of a soft-bodied robot inspired by true slime mold. Biol. Cybern., 102:261–269, 2010. [44] T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. A soft deformable amoeboid robot inspired by plasmodium of true slime mold. International Journal of Unconventional Computing, 7:449–462, 2011. [45] H. Yuasa and M. Ito. Coordination of many oscillators and generation of locomotory patterns. Biol. Cybern., 63:177–184, 1990. 24 Abstract: Morphological Computation is a broad concept that tries to connect the body (or mechanics), brain (or controller) and environment. In this work, we analyze Physical Reservoir Computing as a practical approach to Morphological Computation. Physical Reservoir Computing extends the Reservoir Computing framework that was originally developed in the context of artificial neural networks to physical systems. This has approach provides simple, yet efficient methods to exploit the computational power available in physical dynamical systems. A Reservoir Computing View of Morphological Computation Introduction We know that when the eye sees, all the consequent information is transmitted to the brain by means of electrical vibrations in the channel of the optic nerve. This is an exact analogy with the electrical vibrations which occur in the cable of a television set: they convey the picture from the photocells which see it to the radio transmitter from which it is broadcast. We know further that if we can approach that cable with the proper instruments, we do not need to touch it; we can pick up those vibrations by electrical induction and thus discover and reproduce the scene which is being transmitted, just as a telephone wire may be tapped for its message. Vannevar Bush [2] Vannevar Bush wrote this in his famous article on making knowledge more easily accessible. He was not discussing how body and mind interact, yet this paragraph explains a classical view that the body acts simply as a transmitter of information from the outside world to the brain. We argue that the body is a valuable computational resource and far more than a passive transducer of information. In this manuscript, we introduce our perspective of Physical Reservoir Computing for Morphological Computation. Reservoir Computing is a set of related techniques that originated in the context of recurrent neural networks. Today, the core concepts of Reservoir Computing have been extended beyond artificial neural networks and into the realm of physical systems, hence the name Physical Reservoir Computing. Morphological Computation is a broad term that describes how the body (i.e. the morphology or hardware structure) of a physical agent (robots and animals are typical examples) can simplify control or sensing problems [25]. It is related to the concept of embodied cognition as it relies heavily on the idea that intelligence needs to be grounded in an environment [7, 6, 32]. We will revisit the main concepts of Reservoir Computing and explain how Physical Reservoir Computing has extended this set of techniques from artificial neural networks to physical devices. The view of Morphological Computation developed herein is a practical one, as we focus on how the computational resources in a physical agent can be exploited to simplify control tasks. We do not address the issue of how to build agents that maximize the available computational resources. The main goal is thus to explain how Reservoir Computing provides simple and efficient methods to exploit the intrinsic computational resources of a physical agent. Furthermore, we limit the scope of this work to the practical aspects of Physical Reservoir Computing and do not address philosophical implications. Recalling Vannevar Bush's quote: In the Physical Reservoir Computing approach described herein, the body is not simply seen as a computational unit that relays perceptive information from the world to the brain. The morphology transforms (i.e. performs computations on) the information it perceives and defines how the instructions received from higher control centers result in physical actions. In Physical Reservoir Computing, the term computation is used in the literal sense and the body becomes an analog to a 26 A Reservoir Computing View of Morphological Computation recurrent neural network. We begin this article with an overview of Physical Reservoir Computing and its application to Morphological Computation. We then describe how the original Reservoir Computing concept evolved into Physical Reservoir Computing. Next, we review results obtained by our group on Physical Reservoir Computing for tensegrity robots. Before presenting our conclusions, we discuss the core ideas of Physical Reservoir Computing for Morphological Computation. Physical Reservoir Computing: A Practical View Overview of Reservoir Computing The general concept of Reservoir Computing is to use a dynamical system as a computational black box. Information is fed into the system, which provides random projections of the input and short-term memory. Reservoir Computing allows to efficiently use a large range of dynamical systems to emulate a desired filter1. The dynamical system is left untouched, instead only an observation or output layer is trained. The basic open loop Reservoir Computing method is limited to systems with only short-term memory requirements, the inclusion of feedback connections extends the approach to long-term memory and autonomous signal generation tasks. The clear advantage is that only limited knowledge or control of the dynamical system is needed. This is at the same time the main disadvantage of these methods. Better performance could sometimes be obtained by training the computational substrate. However, Reservoir Computing is an attractive method as it tends to have good performance on various problems and can in general be implemented faster than competing algorithms. Furthermore, it is often not possible to change the internal dynamics of a physical system and a good model may not even be available. Because of these features, it is also an excellent baseline for more involved techniques. In Silico Reservoirs The most common implementations of Reservoir Computing are software based. Three types of software implementations are well known: Liquid State Machines, Echo State Networks and Backpropagation-Decorrelation. However, similar techniques appeared in the literature prior to the introduction of Reservoir Computing, but they were not widely embraced. Echo State Networks were introduced by Jaeger in 2001 [16] and are based on discrete time neurons with a sigmoid activation function. Liquid State Machines were developed by Maass et al. [22] around the same time and provide a more biologically plausible perspective. This technique is typically implemented as a set of (continuous time) differential equations. Finally, Steil presented an efficient single step backpropagation algorithm which uses a Reservoir at its core [29]. As much of 1Without feedback connections, a Reservoir Computing system is limited to the emulation of nonlinear finite impulse response filters. Feedback connections allow infinite impulse response filters to be learned. 27 A Reservoir Computing View of Morphological Computation the knowledge and intuition transfers between these approaches, the general principle of these methods later became known as Reservoir Computing [31]. Win Wout Wres outputreservoir input 1 Wbias Wfb Figure 1: Schematic overview of a Reservoir Computing system. The center part is called the Reservoir and consists of a random recurrent neural network with fixed weights. To train the system to solve a computational task, only the output weight matrix W out is optimized. When the feedback weights W fb are non-zero, the system can be used to solve signal generation and long-term memory tasks. Otherwise, the setup is limited to the emulation of FIR filters, because the dynamics of the Reservoir are generally damped. Figure 1 shows the basic architecture of a standard Reservoir Computing setup with multiple inputs and outputs. The center part is called the Reservoir and consists of a random recurrent neural network. The Reservoir is fixed, meaning that the weights of the internal connections remain constant. As such, it becomes a computational black box, because we cannot change its dynamics and the recurrent nature makes full analysis infeasible. In Reservoir Computing only the output layer W out is trained, while all other weights are kept fixed. The output layer extracts the computations performed by the Reservoir (the black box) and combines them to approximate a set of desired output signals. The Reservoir should have the fading memory property, which means that the network state should become independent of past inputs and network states. Without feedback, a Reservoir Computing system is limited to the emulation of non-linear finite impulse response filters. When fixed feedback connections (W fb ) are added to the system, a Reservoir Computing setup can be also used for long-term memory or signal generation tasks. Physical Reservoirs While artificial Reservoirs have now been studied for well over a decade, there is a recent trend to study the computational or Reservoir properties of physical systems. In fact, the possibility of physical Reservoirs was realized early during the onset of Reservoir Computing systems, with such demonstrations as a bucket of water used as a Reservoir Computing system [11]. The recent developments in physical implementations of Reser28 A Reservoir Computing View of Morphological Computation voir Computing are focused on applications which benefit from the fact that it allows to exploit computational capabilities without precise control or knowledge of all the aspects of the physical system. One domain that has seen a significant influence from Reservoir Computing techniques is optoelectrical and all-optical computing [10, 18, 24]. Furthermore, it was recently shown that a large class of dynamical systems (artificial or physical) inherently have an equal amount of information processing capacity [8]. These are not computations in the Turing sense [30]. Instead, the result from [8] essentially shows how difficult it is to linearly approximate (w.r.t. the quadratic norm) desired transformations of an input stream based on observations of the state of a dynamical systems. One caveat is that a desired type of processing might be present in a physical system, but it can be unfeasible to extract it due to sensor limitations. On the contrary, a system might appear unsuitable for a computational task, while an intelligent encoding can dramatically increase the performance. By this last statement we mean that a dynamical system might for example fail at encoding a desired input transformation as pulses, while it performs optimally when encoding it in the frequency spectrum. In this case, the desired information processing is available in the system, but it is hard to effectively use it. But how is Physical Reservoir Computing any different from centuries-old analog or mechanical computers [15, 9]? The answer is that systems need not be designed to solve the task at hand. The method allows to exploit computations inherently performed by the dynamical system. We believe that the real issue is to maximally exploit useful computations available in a dynamical system. Physical Reservoir Computing provides a means to employ a physical substrate as a computational tool, even if not all its details are fully known, controllable or understood. In the context of compliant robotics, Physical Reservoir Computing sees the body or morphology of the robot as a computational resource. Information from interactions of the robot with its environment are inherently processed (not simply transfered) by the body. The goal of Physical Reservoir Computing for Morphological Computation is to exploit the computations implicitly performed by the robot's morphology. This allows to offload significant parts of certain control problems to the body, which in essence provides free computational resources. Locomotion control of compliant robots is an ideal test case for Physical Reservoir Computing, due to the rich interactions of compliant robots with their surroundings. We mentioned the fading memory property of Reservoirs, which ensures that past input do not indefinitely influence the state of a Reservoir (without feedback). In the case of physical systems, this cannot always be enforced (e.g. a single input value can cause a robot to tumble). Nevertheless, the Reservoir analogy has proven successful in practice. Applications in Tensegrity Robotics In this section, we provide a concise overview of the developments of Physical Reservoir Computing based controllers for compliant tensegrity robots [4, 3]. Related demonstra29 A Reservoir Computing View of Morphological Computation tions of Physical Reservoir Computing in other compliant robotic systems are provided in [34, 23]. Tensegrities (tensile-integrity) are structures in which compressive elements are held together by tensile elements [27]. Due to the specific arrangement of the members, it is possible to make highly efficient use of materials as only axial forces are present (no bending or shear forces). More precisely, one can build free-standing structures consisting of only struts and cables in which no two struts are connected to each other. Forces diffuse throughout the whole system instead of concentrating at the joints, decreasing the risk of failure due to impacts. These properties make them ideal candidates for environments requiring robust and capable robot designs. From a Physical Reservoir Computing perspective, there are a number of reasons to study Morphological Computation aspects on tensegrities rather than on other platforms. Hauser et al. previously introduced a theoretical framework for Morphological Computation based on spring-mass nets [13, 12]. The dynamics of tensegrity structures are similar to those of spring-mass nets and the gap between tensegrity hardware and the theoretical results from Hauser et al. is thus small. Tensegrity robots can be seen as structured soft robots. Indeed, tensegrities offer a global level of compliance, but are composed of discrete elements. These elements can be tuned, very much like the weights in a neural network. The discrete structure is a significant advantage over typical soft robots (e.g. [21]) as it simplifies simulations and the study of the mechanical properties. Compared to other types of compliant robots (e.g. quadrupeds), the most attractive aspect of tensegrities is their pure compliance. Compliance in tensegrities is a global aspect. In most compliant robots, the number of flexible elements or compliant actuators is limited or they are restricted to a small part of the robot. Figure 2 illustrates the basic concept of Physical Reservoir Computing for compliant tensegrity robots. The robot consists of flexible (springs in line with a cable) tensile elements and fixed length compressive members (bars). Sensors are located on the tensile elements and provide (non-linear) state observations. Actuators can modify the rest length (the length at which the force on the member becomes zero) of a subset of the tensile elements. The goal is to find a suitable static, linear feedback controller (W out in the Figure) that connects the sensors directly to the actuator commands. This is analog to how pattern generation tasks are often solved in classic Reservoir Computing. In [4] we introduced online learning methods to solve this problem, under the assumption that the desired motor commands were known. This resulted in robust controllers, capable of maintaining a desired gait. An example of such a controller is given in Figure 3. This Figure illustrates a Physical Reservoir Computing based controller which is able to modulate the desired gait patterns of a simulated 6 bar tensegrity robot by changing a physical input [4]. More precisely, we applied a single input signal by modifying the equilibrium length of two springs in a 6 bar tensegrity robot. The system had to linearly interpolate between two rhythmic signals (3 dimensional) with the same fundamental frequency. The motor signals are a linear combination of (raw) sensor measurements and the Physical Reservoir Computing approach thus effectively uses the computations implicitly performed by the robot dynamics. More precisely, the motor signals are computed by 30 A Reservoir Computing View of Morphological Computation f passive spring actuated spring sensor motor l0 l0 = Wout f Figure 2: The basic concept of Physical Reservoir Computing for compliant tensegrity robots as introduced in [4]. The thick red lines are fixed length bars. The thin green lines are passive tensile elements and the dashed blue lines are actuated tensile elements. The motors on the actuated elements change the rest lengths of the tensile members. The goal is to find an optimal static feedback controller to autonomously generate desired motor signals. Figure has been taken from [4]. time (5 s) RLS output target signal input signal ac tu at or s ig na ls Figure 3: Modulating gait patterns for a compliant tensegrity robot through Physical Reservoir Computing. In this example, an input signal (the green bottom green line) is injected into the robot by modifying the rest length of 2 actuated tensile members (see Figure 2). At the same time, a static feedback controller is trained (defined by the matrix W out in Figure 2) which combines the available sensor data to approximate the desired actuator signals (shown in in dashed red lines). The method uses the well-known Recursive Least Squares (RLS) algorithm during training. The input signal indicates how the motor signals should be modulated. The black lines show the actuator signals during testing, which closely match the desired ones. Figure has been taken from [4]. 31 A Reservoir Computing View of Morphological Computation multiplying the vector of sensor data with the learned weight matrix W out . This example shows one of the general principles of Physical Reservoir Computing. We exploit the dynamics of a system to perform quantifiable computational tasks. Indeed, we can measure how well the robot can approximate the desired signals. In Physical Reservoir Computing, the computational black box (the tensegrity robot in this case) is used as-is and the available sensor data is optimally combined to solve a task. This is more restricted than general Morphological Computation as we are not studying how the robot is solving a task. We are simply employing a dynamical system as a computational resource, the performance of which can be quantified. For the example presented herein, the Recursive Least Squares (RLS) algorithm was used to train the system during 400s with random inputs. As can be seen in the Figure, this is sufficient to accurately generate the desired signals based on the sensor data. At approximately two thirds of the sample, there is a noticeable difference between the desired and observed signals. The (RLS) output signal mismatches the target because of a sudden large drop of the input signal. The reason for this is that the output is based on the state of the physical system and thus intrinsically smooth. As such the feedback will naturally interpolate between target signals, which results in a robust controller. To validate the simulation results, we developed a hardware tensegrity robot platform nicknamed ReCTeR (Reservoir Compliant Tensegrity Robot) [5, 1]. This robot is shown in Figure 4. ReCTeR is a small (1m diameter), untethered, lightweight (1.1kg) and underactuated (6 DC motors) robot. Its design is based on the common tensegrity icosahedron (6 struts, 24 tensile elements). ReCTeR can fold, deploy and roll and has a battery life of over 30min with all systems active. This makes it possible to validate the Physical Reservoir Computing techniques using experimental results obtained on this hardware robotic platform. Despite its low weight, ReCTeR is equipped with a large amount of sensors to enable feedback control for Physical Reservoir Computing. Future Developments Our demonstrations of Physical Reservoir Computing for compliant robots made use of a static output/feedback layer (W out is fixed after training). While this already allows for capable and flexible controllers, some oscillator modulations are hard (e.g. frequency tuning). Recently, new techniques have been developed to tune the dynamics of Reservoir Computing based oscillators [33, 17]. Similar to the original Reservoir Computing methods, we anticipate that these methods will also soon be implemented on compliant robots. This will further increase the flexibility of the Physical Reservoir Computing approach and allow direct interaction with higher-level controllers. More precisely, layered architectures can be created in a straightforward way. The low-level dynamics are handled by the Physical Reservoir Computing approach, while the modulations (e.g. frequency, gait, . . . ) are controlled by a higher level system. Another research direction which is yet to see full development are reward modulated2 Hebbian plasticity based learning rules [14, 28]. This type of learning rules moves be2In this case, modulation refers to neuromodulation. The weight updates of a Hebbian-like plasticity rule are modulated by a reward signal. 32 A Reservoir Computing View of Morphological Computation Figure 4: The ReCTeR tensegrity robot which allows to validate simulation results. ReCTeR is a small, untethered, lightweight and underactuated robot capable of folding, deploying and rolling. It is equipped with a large amount of sensors to enable feedback control. (image credits: NASA/Eric James) yond Reservoir Computing because the internal dynamics can also be trained. Reward modulated Hebbian plasticity as studied in [4], [14] and [20] is not strictly a Reservoir Computing approach, precisely due to the changes to the internal dynamics. However, it is closely related, because the basic system can still be a black box, but we can now change some of its parameters (i.e. a tunable black box) without knowledge of the influence of each parameter. The main advantage of this type of learning rules is that no knowledge of the optimal control signals is required. Instead, reward signals based on global performance measures (e.g. the trajectory of an end-effector) are sufficient to learn feedback controllers or even modify the internal dynamics. Discussion: The Core Ideas There are two main ideas we hope to bring across. All Things Compute Every physical system is an information processing device. At any scale, physical systems interact with each other and in fact one often characterizes or defines a system based on how it interacts with other systems. Morphological Computation mostly focuses on robotics and biological systems and tries to explain how the physical organization of an agent influences its behavior and intelligence. As this is a broad and at times ill-defined concept, it is ultimately useful to limit the scope. Physical Reservoir Computing is a practical approach to a subset of Morphological Computation. Borrowing from rich results from recurrent neural networks, we attempt to exploit the computational resources of physical dynamical systems. Morphological 33 A Reservoir Computing View of Morphological Computation Computation and Physical Reservoir Computing are thus not equivalent. Indeed, Morphological Computation also studies how a robot can perform a specific type of computation by using its morphology. For example, a Braitenberg vehicle is an example of Morphological Computation because the sensor morphology clearly defines the behavior of the robot. Physical Reservoir Computing does not explain such behaviors. In one of their original papers on the topic, Pfeifer and Iida put forward the question [26]: We would like to be able to ask "How much computation is actually being done?" This is a hard question as it requires a good measure and a definition of computation that makes sense in the context of specific application. Physical Reservoir Computing takes a pragmatic approach to answer such questions as it is ultimately concerned with how well a given computational unit can emulate a set of desired signals. An exemplar research question that can be addressed with Physical Reservoir Computing is to what extent terrain properties can be extracted from the available sensor data in a mobile robot. But as we discussed earlier, Physical Reservoir Computing is not limited to robotics and biological systems. The principle is actively being studied for electronic and photonic system [10, 18, 24]. Thus Physical Reservoir Computing and Morphological Computation are distinct subjects with an interesting partial overlap. Why Link Reservoir Computing and Morphological Computation? Why is Physical Reservoir Computing an compelling field of study when it is restricted to a subset of Morphological Computation? The answer to this question is that Physical Reservoir Computing provides a simple and quantifiable set of tools with minimal assumptions about the computational substrate. We do not attempt to explain how computations are performed, but focus on how to use them. Physical Reservoir Computing is thus a practical and restricted approach, rather than a full theory of Morphological Computation. It has direct applications in robotics (e.g. robust feedback controllers) and has now been successfully demonstrated on multiple hardware platforms. Conclusions Physical Reservoir Computing is a recent effort to extend a set of learning methods originally targeted at recurrent neural networks into the realm of physical systems. This work unfolded our vision on how Physical Reservoir Computing relates to Morphological Computation. By following a practical approach, we argued that Reservoir Computing provides a capable set of tools to exploit the computations inherently available in physical dynamical systems. We provided examples from the compliant robotics field as it is in this domain that we see the most direct application of Physical Reservoir Computing based Morphological Computation. 34 A Reservoir Computing View of Morphological Computation In particular, we focused on tensegrities, a type of highly compliant robots which are closely related to the spring-mass nets considered by Hauser et al. in their theoretical foundation of Morphological Computation. Tensegrity robots are a convenient tool for the study of Morphological Computation for multiple reasons. They can be fully compliant (in the sense that they do not have rigid joints) and highly robust. This has the advantage that the controls can be less precise and noisy and tensegrities are thus an ideal test case for learning based controls. Furthermore, they can be seen as structured soft robots and Physical Reservoir Computing based control methods are therefore timely, considering the recent emergence of the soft robotics field [19]. The main advantage of Physical Reservoir Computing approach is the fact that parts of the dynamical system can essentially be black boxes. This is crucial to many practical applications as physical agents in general do not have access to full models of their environment or proper dynamics. We anticipate that some recent developments for in silico reservoirs will soon be transferred to robotic systems. This will allow for hierarchical control in which the low-level "black boxes" are robustly handled by Physical Reservoir Computing based controllers. Higher level controllers then modulate the low-level systems. The take-home message is that Physical Reservoir Computing provides a set of simple, yet efficient tools to simplify the use of complex dynamical systems such as compliant robots. It assumes little or no knowledge about the internal dynamics of the system at hand and instead builds upon a well-established learning method for recurrent neural networks. Acknowledgments The research leading to these results has received funding from the European Community's Seventh Framework Programme FP7/2007-2013-Challenge 2 Cognitive Systems, Interaction, Roboticsunder grant agreement No 248311-AMARSi. Ken Caluwaerts was supported by a doctoral fellowship of the Research Foundation Flanders (FWO). Bibliography [1] J. Bruce, K. Caluwaerts, A. Iscen, A. P. Sabelhaus, and V. SunSpiral. Design and evolution of a modular tensegrity robot platform. In IEEE International Conference on Robotics and Automation, ICRA, 2014. [2] V. Bush. As We May Think. Atlantic Monthly, 176(1):641–649, March 1945. ISSN 1072-5520. doi: 10.1145/227181.227186. URL http://www.theatlantic.com/doc/ 194507/bush. [3] K. Caluwaerts and B. Schrauwen. The body as a reservoir: locomotion and sensing with linear feedback. In 2nd International Conference on Morphological Computation, pages 45–47, Venice, 2011. 35 A Reservoir Computing View of Morphological Computation [4] K. Caluwaerts, M. D'Haene, D. Verstraeten, and B. Schrauwen. Locomotion without a brain: physical reservoir computing in tensegrity structures. Artificial Life, 19 (1):35–66, 2013. doi: 10.1162/ARTL_a_00080. Special Issue on Morphological Computation. Editors: H. Hauser, H. Sumioka, R.M. Füchslin, R. Pfeifer. [5] K. Caluwaerts, J. Despraz, A. Işçen, A. P. Sabelhaus, J. Bruce, B. Schrauwen, and V. SunSpiral. Design and control of compliant tensegrity robots through simulation and hardware validation. Journal of The Royal Society Interface, 11(98), 2014. doi: 10.1098/rsif.2014.0520. [6] H. J. Chiel, L. H. Ting, O. Ekeberg, and M. J. Z. Hartmann. The brain in its body: motor control and sensing in a biomechanical context. Journal of Neuroscience, 29 (41):12807–14, 2009. ISSN 1529-2401. [7] A. Clark. Being there: Putting brain, body and world together. MIT Press, 1997. [8] J. Dambre, D. Verstraeten, B. Schrauwen, and S. Massar. Information processing capacity of dynamical systems. Scientific Reports, 2(514), 2012. [9] D. de Solla Price. A history of calculating machines. IEEE Micro, 4(1):22–52, 1984. [10] F. Duport, B. Schneider, A. Smerieri, M. Haelterman, and S. Massar. All-optical reservoir computing. Optics Express, 20(20):22783–22795, 2012. [11] C. Fernando and S. Sojakka. Pattern recognition in a bucket. In Advances in Artificial Life, pages 588–597. Springer, 2003. [12] H. Hauser, A. J. Ijspeert, R. M. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological Cybernetics, 105:355–370, 2012. ISSN 1432-0770. doi: 10.1007/s00422-012-0471-0. [13] H. Hauser, A. J. Ijspeert, R. M. Füchslin, R. Pfeifer, and W. Maass. The role of feedback in morphological computation with compliant bodies. Biological Cybernetics, 106(10):595–613, 2012. [14] G. M. Hoerzer, R. Legenstein, and W. Maass. Emergence of complex computational structures from chaotic neural networks through reward-modulated hebbian learning. Cerebral Cortex, 24:677–690, 2012. [15] G. Ifrah. The Universal History of Computing: From the Abacus to Quantum Computing. John Wiley & Sons, Inc., 2000. ISBN 0471396710. [16] H. Jaeger. The "Echo State" approach to analysing and training recurrent neural networks. Technical Report GMD report 148, German National Research Center for Information Technology, 2001. [17] H. Jaeger. Controlling recurrent neural networks by conceptors. arXiv preprint arXiv:1403.3369, 2014. 36 A Reservoir Computing View of Morphological Computation [18] L. Larger, M. Soriano, D. Brunner, L. Appeltant, J. M. Gutiérrez, L. Pesquera, C. R. Mirasso, and I. Fischer. Photonic information processing beyond turing: an optoelectronic implementation of reservoir computing. Optics Express, 20(3):3241– 3249, 2012. [19] C. Laschi and M. Cianchetti. Soft robotics: new perspectives for robot bodyware and control. Frontiers in Bioengineering and Biotechnology, 2(3), 2014. [20] R. Legenstein, S. M. Chase, A. B. Schwartz, and W. Maass. A reward-modulated Hebbian learning rule can explain experimentally observed network reorganization in a brain control task. Journal of Neuroscience, 30(25):8400–8410, 2010. [21] H.-T. Lin, G. G. Leisk, and B. Trimmer. GoQBot: a caterpillar-inspired soft-bodied rolling robot. Bioinspiration & Biomimetics, 6(2):026007, 2011. [22] W. Maass, T. Natschlager, and H. Markram. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531–2560, 2002. [23] K. Nakajima, H. Hauser, R. Kang, E. Guglielmino, D. Caldwell, and R. Pfeifer. Computing with a muscular-hydrostat system. In IEEE International Conference on Robotics and Automation, ICRA, pages 1504–1511, May 2013. doi: 10.1109/ ICRA.2013.6630770. [24] Y. Paquot, F. Duport, A. Smerieri, J. Dambre, B. Schrauwen, M. Haelterman, and S. Massar. Optoelectronic reservoir computing. Scientific Reports, 2, 2012. [25] R. Pfeifer and J. Bongard. How the body shapes the way we think: A new view of intelligence. MIT Press, 2007. ISBN 9780262162395. [26] R. Pfeifer and F. Iida. Morphological computation: Connecting body, brain and environment. Japanese Scientific Monthly, 58(2):48–54, 2005. [27] R. E. Skelton and M. C. de Oliveira. Tensegrity systems. Springer, 2009. [28] A. Soltoggio and J. J. Steil. Solving the distal reward problem with rare correlations. Neural Computation, 25(4):940–978, 2013. [29] J. J. Steil. Backpropagation-Decorrelation: Online recurrent learning with O(N) complexity. In IEEE International Joint Conference on Neural Networks, IJCNN, pages 843–848, 2004. [30] A. M. Turing. On computable numbers, with an application to the entscheidungsproblem. Proceedings of the London Mathematical Society, 42:230–265, 1936. [31] D. Verstraeten, B. Schrauwen, M. D'Haene, and D. Stroobandt. 2007 special issue: An experimental unification of reservoir computing methods. Neural Networks, 20 (3):391–403, 2007. 37 A Reservoir Computing View of Morphological Computation [32] M. Wilson. Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4): 625–636, 2002. ISSN 10699384. [33] F. wyffels, J. Li, T. Waegeman, B. Schrauwen, and H. Jaeger. Frequency modulation of large oscillatory neural networks. Biological Cybernetics, 108(2):145–157, 2014. [34] Q. Zhao, K. Nakajima, H. Sumioka, H. Hauser, and R. Pfeifer. Spine dynamics as a computational resource in spine-driven quadruped locomotion. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pages 1445–1451, Nov 2013. doi: 10.1109/IROS.2013.6696539. 38 Abstract: In this chapter we explore the question of how to scale up the phenomenon of morphological computation to complex adaptive systems. We suggest that an important inspiration may come from the concept of embodied appraisal in human emotion: how physiological states may come to "represent" qualitative features of the interaction. Emotion-based robotics and morphological computation therefore share a common ground, which we explore in this chapter, and venture that insights from each area may prove vital for the development of the other. Our aim is to explore and clarify the potential of this cross-fertilization. Deep into Morphology: Emotions and Functional Structure Introduction The field of morphological computation (as applied to robotics) faces the same challenge as autonomous robotics in general: how to scale up concrete insights on adaptive mechanisms towards the design of full-blown autonomous systems. The field has shown, among other claims, that morphology can simplify computational problems, that all cognition must be framed in the context of sensory-motor coordination, that behavioural diversity may be due to morphological characteristics rather than to a central controller, and that morphology and control should be co-designed [22]. Exploitation of these insights in robot design can be clearly demonstrated in simple cases where there is little gap between sensory-motor coordination and cognition – these are coupled phenomena. It might be harder to establish links to the full range of cognitive processes a complex autonomous system may engage in. The intuition is that all cognition must be ultimately embodied. But one thing is to say that the body will always be relevant, and the other that it will hold the key answers to the question of cognition. How we bridge the gap between low-level mechanisms in a system and models of its cognitive organization remains a great challenge. An interesting approach to this problem comes from an seemingly unrelated area: robotic models of emotion. A great influence here has been neuroscientist Antonio Damasio's thesis that cognition should not be considered independent of emotion, but built upon it [3]. "Some of the machinery of the immune system and of metabolic regulation is incorporated in the machinery of pain and pleasure behaviours. Some of the latter is incorporated in the machinery of drives and motivations (most of which revolve around metabolic corrections and all of which involve pain or pleasure). Some of the machinery from all the prior levels- reflexes, immune responses, metabolic balancing, pain or pleasure behaviours, drives-is incorporated in the machinery of the emotions-proper."[4, p.36]. Emotion is defined as "a specifically caused transient change of the organism state", a global change in bodily function of all systems, including the nervous system [4, p.153]. Above emotion comes feeling, "the representation of that transient change in organism state in terms of neural patterns and ensuing images" [4, p. 282]. Cognitive capacities, at least the sort we consider enter the spectrum of consciousness, rely on this capacity for feeling. For instance, "somatic markers" are hypothesized to link the internal milieu and current perceptions to give value to what is perceived. Researchers in this line have thus advocated for a "multi-tiered affectively embodied view of mind", where emotion and cognition emerge from lower levels of organization [20].There is no general agreement regarding the tiers that form cognition, although a clear pattern emerges. Ortony [19] suggests reactive, routine, and reflective information processing; Sloman [27], taking an architecture approach distinguishes between reactive, deliberative, and meta-management levels. See [28] and [29] for an excellent review of the question of levels and current approaches. Can this approach be successful in helping to scale up morphological computation? A point of coincidence between morphological computation and emotion is the commitment to embodiment. The importance of the body has been recognized by virtually every single 40 Deep into Morphology: Emotions and Functional Structure author that took emotion into consideration, from Aristotle to Descartes and modern psychologists. "What kind of an emotion of fear would be left, if the feelings neither of quickened heart-beats nor of shallow breathing, neither of trembling lips nor of weakened limbs, neither of goose-flesh nor of visceral stirrings, were present, it is quite impossible to think". [15, p.193]. A lot happens to the body in emotion, processes that play an important role in determining the behavioural outcome. Emotion is a key element of the organization of embodied systems, in ways that are not fully understood. There has nevertheless been a lack of cross-fertilization between these two areas, mainly due to a different notion of embodiment. In the case of morphological computation, the body is seen as the means to sensory-motor coordination – while in emotion-based robotics, as we shall see, the body is considered a source of homeostatic signals, such as hormones. Despite this difference, the two fields may be brought closer, and this may provide key answers to problems they both face. In short, a true emotional system is not just monitoring internal states, but assessing the dynamics of the interaction it is engaged in, the tendencies of its behaviour as they project over time. We must take into account not only the dynamics of internal signals, but the influence these have upon the dynamics of interaction, which is very much under the scope of the field of morphological computation. On the other hand, to understand how morphological computation becomes an essential component in an architecture of the mind, we must deal with qualitative aspects of interaction and how they are perceived by the agent in whatever way they matter. This, in turn, leads to the phenomenon of emotion. Morphological Computation and Signal Systems In this section we develop a working definition of morphological computation, with its main role to guide our own analysis of embodied models of emotion. It is not an overreaching definition, as it is restricted to the area of autonomous systems. In short, we will call morphological computation any process relevant from an information-theoretic perspective that relies on bodily systems whose primary function is not to process signals, and whose operation is essential for the computational process in question. In robotics, morphological computation is sometimes contrasted to centrally processed control (e.g. [12], [22]). We know, from an information-theoretic perspective, that any process occurring in an agent's body, and even its surroundings, can present regularities that are essential for understanding the information-flows. There are two theses regarding morphological computation. The first is that sensory information are not bare signals for a cognitive system to data mine. Signals are already structured by morphology, and dynamically constituted in sensory-motor coordination [22]. This is true for all systems that take in information through sensors and effect changes upon the environment, however they may be constructed. The design of a system without taking into account this fact is simple "bad engineering" [26]. One of the tasks of the field of morphological computation is to try to develop techniques and methods that bring this feature to the forefront of the design 41 Deep into Morphology: Emotions and Functional Structure process of autonomous robots. The second claim, which actual significance is harder to grasp, is that cognitive processes at any level are fully dependent on morphological computation. This can be shown in very simple agents, whether these are animals or robots, and their cognitive capacities are very much determined by their morphologies. This runs against the view that thinking is done exclusively in the brain, or by a central computer. Nevertheless, the theory of morphological computation cannot be complete until it understand how advanced cognitive systems can base their cognitive processes on the body. This raises the question, is a system less embodied for having a brain or relying on a computer? The answer to this question might be less obvious that it appears. Ultimately, it brings us to the question whether there is a qualitative difference between processes that may be called embodied and others that can be say to belong to a "detached central controller." From a control engineering perspective, disembodiment simply makes no sense -a controller that is capable of controlling a body with all the dynamical complexities of its interaction with the environment is always embodied, no matter how it is implemented. In a sense, the difference accounts for having a centralized versus a fully distributed controller. While some non-functional1 differences may appear, both approaches offer similar control capabilities. In practical terms, quasi-metaphysical discussions around what counts as embodied or disembodied are ultimately futile. There are not disembodied controllers -as far as they are really controlling. There is nevertheless a fact that cannot be underestimated: both in artificial and natural systems there are subsystems whose primary function is to generate, carry or transform information flows. This is essential for any notion of autonomy beyond reactive agency. This signalling function plays a great role in control and the functional coordination of different subsystems. The role of a specialized signalling subsystems is not to override computational processes at a morphological level, but to coordinate such processes, and ultimately to exploit the regularities that they provide. A central controller thus becomes a key element of embodiment, insofar it is in charge of the functional arrangement of subsystems. The whole bodily organization, both in its structural and functional aspects, is what we call morphology. Insofar morphological computation emerges from the operation of the system as a whole, e.g. in sensory-motor coordination, it cannot be assigned to a particular subsystem, but to the functional integration of all subsystems, which may rely on a specialized informational subsystem. Therefore, in a truly autonomous system, there can be no distinction between morphological and non-morphological computation, except by appealing to the functional difference between systems whose primary function is informational, and those whose is not. There are two main information signal systems in humans: the nervous system and the "endocrine system." Each can be seen to control different aspects of behaviour generation, influencing each other and controlling the behaviour of other systems (e.g. muscular, respiratory, digestive or sensory systems). Emotional responses emerge form this organisation. Functional changes like muscle tone, heart beat, level of adrenaline, etc., affect 1 As used in computer science and not in a philosophical sense. 42 Deep into Morphology: Emotions and Functional Structure the dynamics of interaction in ways that may be adaptive. Emotions can be defined as a change in action readiness that is facilitated through physiological changes that affect the function of all subsystems [9]. These systems may play an important role in the processing of signals, and in fact they do, but this is not their primary function. This has led us to argue here and elsewhere ([14], [13]), that emotion emerges from the control of morphofunctionality. The functional reconfiguration of all subsystems allows the agent to control qualitative aspects of the dynamics of interaction with the environment. This role in controlling the body can ground in physiological states certain computational functions. As we see in the next section, this is called in emotion theory embodied appraisal. Relevance of Robotic Models of Emotion for Morphological Computation: Embodied Appraisal In this section we show that what emotion theorists call embodied appraisal is, under all perspectives, a phenomenon of morphological computation. Appraisal is the process through which an agent is capable of assessing and responding to situations that are seen as relevant to its concerns. Cognitivist approaches tend to believe that appraisal should have propositional content, yet perceptual theories of emotion claim that appraisal is primarily an embodied phenomenon. Prinz presents the most concise hypothesis in this direction, claiming that our bodies are capable of representing agent-environment relationships through bodily states [25, p.45]. His arguments is that "Emotions are somatic, but they are also fundamentally semantic: meaningful commodities in our mental economies" [24, p.45]. In order to support this thesis, Prinz relies on Dretske's notion of representation. According to this theory, a state has intentional content if it has the function of being reliably caused by something, which it comes to represent [5, 6, 8]. It is in virtue of a causal relation that a state (whether a brain state or a physiological state) may have content. "Emotions are . . . perceptions of changes in our somatic condition. But, ironically, they are also appraisals. Let us define an appraisal, not as an evaluative judgment, but as any representation of an organism-environment relation the bears on well-being. Evaluative judgments can serve as appraisals, but they are not alone. If a non-judgmental state represents an organism-environment relation that bears on well-being, it too will count as an appraisal on this definition. My suggestion is that certain bodily perceptions have exactly this property. They represent roughly the same thing that explicit evaluative judgments represent, but they do it be figuring into the right causal relations, not by deploying concepts or providing descriptions. Our perceptions of the body tell us about our organs and limbs, but they also carry information about how we are faring." [24, p.57]. An emotional agent is thus one that is designed to host such a causal connection, and is capable of exploiting it for adaptive purposes. The causal link must connect, on the one hand, physiological changes and the phenomenon of arousal, and on other, information relevant to assess the emotional content of a situation. To denote this class of situations 43 Deep into Morphology: Emotions and Functional Structure Prinz borrows the notion of core relational theme2 from Lazarus' cognitive-motivationalrelational theory of emotion ( Lazarus 1991). To say that emotions are relational is to ground them in the relationship between the agent and the environment, and suggests that emotions always involve an interaction between the two [16]. The stress on the interactive nature of emotional behaviour is important, because what emotions assess is not the situation per se, but the relevance it has in the agent's eyes for its own concerns. In appraisal "in the eyes of the agent" is an essential clause, not because the access to knowledge is limited, but because the nature of the assessment, which primary aim is not objectivity but relevance. In other words, the aim of fear is not to be certain about the degree of danger present in a situation, but to avoid or resolve any situation that may appear to contain some indication that this may be possible. This feature of emotional judgments has a lot to do with the role they play in the organization of emotional behaviour, which is far from being a simple phenomenon. Often, the folk psychology view is that emotions are stereotypical patterns of behaviour, because we can easily observe them in others when they occur, and are ever present in cinema and TV. In every-day life though, emotional behaviour is very seldom stereotypical – rather its aim is to modify the relationship with the environment at large [9]. This constitutes a situated perspective on emotion, which considers essentially the dynamical coupling between agent and environment, which influences and is influenced by the unfolding of emotion. "In traditional models of emotional appraisal, the organism receives information from the environment and uses it to determine the emotional significance of the situation that confronts it. In contrast, the situated perspective envisages organisms "probing" their environment through initial emotional responses, and monitoring the responses of other organisms to determine how the emotion will evolve" [11]. In summary, our take on the theory of embodied appraisal can be summarized in three statements: • Physiological states can in some cases be used as representations of certain situations, in virtue of some reliable causal connection. • The content of such representations refers to the relevance the situation has, in the agent's eyes, to its own concerns. This is a special type of cognitive phenomenon, in regard to the cognitive features of its content (verifiability, objectivity, etc.) • Appraisal serves the generation of emotional behaviour, that is, the capacity of the agent as a whole to modify and adapt how it relates to the relevant features of the environment. The embodied appraisal theory defends that certain agents are designed to support a dynamic coupling with its environment, in ways that certain qualities of the relationship may be traced by monitoring states of its physiology. Such qualities point to the relevance 2 A core relational theme is a description of the type of situations that may give rise to a certain emotion. As an illustration, anger is said to be caused by situations that constitute "a demeaning offense against me and mine" 44 Deep into Morphology: Emotions and Functional Structure of situation for the agent: whether the situation pose a threat, a opportunity, an obstacle, and so on. Appraisal is thus supported by the causal grounding of such processes in relational features. Although the theory is plausible at a conceptual level, it is far from being a wellestablished scientific theory. The main reason is that it does not specify what the reliable causal relationships are. Emotion theory is developed largely through the observation of the human emotional life, which implies an extremely complex organisms immersed in intricate cultural and social systems, and subject to phenomenological experience and linguistic articulation. Hypotheses about the causal connection that grounds embodied appraisal are necessary sketchy and loosely founded. Robot models can become an useful tool to develop theories about embodied appraisal [2]. In the next section we present two approaches to modelling emotion in robots. The first one, that we call the homeostatic approach, seeks to ground embodied appraisal in a form of organization that occurs at different levels. The second one, that we promote here, is the morphofunctional approach, which considers embodied appraisal to be grounded in a control loop that manages morphological control variables. Robotic Models of Emotion The Homeostatic Approach Robotic models of emotion have attempted to pin down the causal relationship between states of activation of physiological systems and emotional situations. The main hypothesis (consistent with Damasio's approach) is that there is a level of organisation that is in charge of homeostatic regulation, maintaining the balance between systems that ensures well-functioning [28]. Whatever cognitive structures come on top, they should be based on bodily function at this homeostatic level. Robotic implementations often make use of simulated hormonal systems [1], energy systems [17], or drives (e.g. hunger) [21], to model the type of signals that may be attributed to homeostatic regulatory systems. This type of work has shown that motivational states rooted on embodiment can play an important role in shaping action selection. It is argued that by introducing internal signals coming from the actual needs of the system, the process that allows the system to become aware of the relevance of the situation for its concerns is replicated. While it does make sense to introduce motivations at this level, results fall short to grasp the idea of embodied appraisal as stated by Prinz. His hypothesis requires a causal link that we can rely on to say: if this internal state is so and so, then the situation must be of this sort. Embodied appraisal means the capacity to evaluate the qualities of interaction. Intrinsic motivations do not to seem close to fulfil the causal link demanded by the embodied appraisal thesis, because it is not clear about the connection between motivational states and the actual dynamical relationship with the environment. The set of tools and theories developed by the field of morphological computation is aimed at understanding how dynamic coupling is grounded in morphology. It thus should be considered when facing the task of understanding embodied appraisal and 45 Deep into Morphology: Emotions and Functional Structure emotional behaviour. What is lacking in current emotion models can also be identified by looking back at our working definition of morphological computation, and ask whether the homeostatic approach can be considered morphological computation. Homeostatic mechanisms are often simulated in artificial endocrine or energy systems, whose primary function is to generate signals, and they do not have a causal link to the space of sensory-motor dynamics and interaction with the environment, beyond motivating action selection. Without such link, we argue, the causal grounding of embodied appraisal is impossible. Whilst standard robotic models of emotion (using homeostasis) attend to internal states, they ignore the coupling between body and world. The Morphofunctional Approach The morphofunctional approach presented here aims to bridge the gap between emotion models and the methods found in morphological computation, and will allow us to present a specific hypothesis about how embodied appraisal is possible. We have presented this theory in two recent papers, giving details about the biological inspiration of the approach [14], and a control systems approach to the issue [13]. The focus here is on the question of embodied appraisal. As stated above, the challenge is to find what causal links may ensure a representational status of physiological states towards the awareness of core relational themes. If we first look at emotional biological systems, there are two aspects of such a response that we must take into account. A lot of research is focused on the execution of particular goal-oriented activities that are necessary to cope with the challenges of the situation. A common example is that of fear. When an agent appraises a situation as challenging some of its concerns, there may be a phenomenon that emotion theorists have called "control precedence"[9]. The occurrence of an emotion may often lead to the interruption of current activities, overriding other considerations about long-term effects of the actions, and the immediate execution of particular actions, whether they are reflex-like or deliberative. This phenomenon is of extreme importance, but we argue much can happen in emotion before control precedence is effective – which ultimately may be explained as a consequence. This has been conceptualized by researchers with the concept of action readiness. A change in action readiness is not a change in a behavioural plan, but a change of the underlying configuration adapted to engage in different cognitive and behavioural processes. This can be related to the concept of morphofunctional machines: "devices that can change their functionality not only by a change in (neural) control but by modifying their morphology" [23]. Physiological changes are the primary mechanisms biological agents have for controlling their morphofunctionality, and they do so in emotion to adapt to ongoing situations. Although robots have no biological physiology, the functional aspects of the morphology of any robotic system are determined by some parametric configuration. Parameters are essential to define the behaviour of every single device and subsystem of the robot. The tendency in engineering is to fix such parameters, to make the behaviour of the system as a whole more predictable and allowing the use 46 Deep into Morphology: Emotions and Functional Structure of dynamical system models to control interaction. But the capacity to change such parameters is of great engineering value, as is demonstrated in variable structure control systems [7]. In [13] we argue that controlling action readiness may be considered a form of variable structure control – changes that cause the dynamic structure of the interaction to vary. In biological agents we find two subsystems, the nervous and endocrine systems, whose primary functions are to carry signals towards system control. In a simplistic way we could say that while the nervous system is responsible for action control, the endocrine system and the autonomic nervous system control action readiness. This is simplistic because both tasks largely overlap when integrated into a general control strategy. We can even say that the nervous system controls the endocrine system, and viceversa. Emotion can be considered to emerge from this control setup. In the morphofunctional approach the core of emotion lies in the changeable dynamics of interaction and the mechanisms an agent has in order to manage them adaptively. When trying to grasp the causal grounding of appraisal in physiological states, we should consider the implications physiological changes have on the dynamics of interaction. It is in this setup that by controlling action readiness, information from the performance of the agent's subsystems (in the form of feelings or body maps in Damasio's theory[3]), becomes a central player in the cognitive organisation of behaviour. This is ensured by a bidirectional causal link between physiological states and agent-environment relationship through the dynamics of readiness. Much research needs to be done to demonstrate the connection between morphofunctional control and emotion, and its value for the development of autonomous robotics. The hypothesis for such work is that through morphofunctional control, a system may be capable of performing embodied appraisal and reconfigure its morphofunctional state appropriately, to provide through changes in action readiness changes in its relationship with the environment. In the following section we present an architecture for embodied appraisal based on the notion of morphofunctional control as well as action control, our proposed approach. An Architecture of Embodied Appraisal When trying to apply the previous analyses to the construction of artificial systems, it is necessary to identify the core elements of the embodied appraisal model to help to understand the relation and architectural traits that enable the adaptive behaviour that emotional morphofunctionality can provide. Note that the critical issue is the representation of agent-environment relational states and how these representations do lead to morphological changes in the agent. The architecture of embodied appraisal is a systemic architecture that is explanatory of the natural phenomena (aggregating theories from Prinz, Lazarus, Frijda and Scherer) and operational for the construction of adaptive artefacts. Emotion-driven morphofunctionality provides an increased capacity for system adaptivity based on the dynamical re-instantiation of relational modes of agent-environment interaction. The key issue in 47 Deep into Morphology: Emotions and Functional Structure this model is that emotions are driven by the perception and appraisal of the state of these relational modes. Physical Level Functional Level Agent State Sensory Subsystem Fast (neural) Motor Subsystem Body Environment Appraisal System Signal Systems Slow (hormonal) Sensory-motor coordination Emotion Signals Core relational theme Realized into Perception System Action Controller Morphofunctional Controller Realized into Configurations Figure 1: A block diagram of the elements in the morphofunctional approach to emotions. Thick grey arrows describe the flow of the functional level metacontrol. Dashed arrows express realization relations between the abstract functional organization level and the physical agent realization. The double headed arrow describes the core relational theme addressed in this chapter. Figure 1 shows a block diagram3 of the principal elements -blocksthat sustain the implementation of such a mechanism. Some relations are not shown to simplify and make the diagram more readable. The physical reality of the autonomous system is composed of the agent body and its environment. In the agent's body we can observe different subsystems: a sensory subsystem, a motor subsystem and two different signal systems. The functional level is represented by the aggregation of perceptors, models and actors. We could say that there is a correlation between the functions and subsystems (perception is implemented by sensory systems, action control by fast signal systems, morphofunctional control by slow signal systems, etc.). This mapping is nevertheless not rigorous, because functions can only be implemented by the operation of the system as a whole. Embodied appraisal, for instance, cannot be easily mapped to any subsystem. The organization of the agent is governed by a morphofunctional controller that acts upon the agent subsystems and their inner relations to instantiate specific agent configurations. The instantiation of a specific configuration enacts a mode of relation of the 3 Using the SysML syntax from [18]. 48 Deep into Morphology: Emotions and Functional Structure agent with its environment, a specific action readiness, to realize specific relational themes that are adequate for the attainment of the agent goals in specific world situations. The activation of the different configurations is carried out by the emotion system (appraisal system and morphofunctional controller). The appraisal of current relational theme derives from the perception system, the controller models and the state of activation of the different configurations. Appraisal triggers emotional signals that drive the morphofunctional agent reconfiguration processes, which feedback into the appraisal process. The emotional system constitutes hence a metacontrol layer that governs the state of the agent subsystems to instantiate the adequate configurations for action readiness. The aim of the morphofunctional approach is not to implement emotions, but the underlying processes from which they emerge. "Instead of talking about emotions, one might instead describe streams of concurrent and interacting ongoing processes: appraisals that last and change, that activate processes of action readiness that generate action preparations and overt actions that act back upon appraisals, that all vary in degree of activation, and that each have different time courses and different moments at which they die down."[10] Appraisal of situations and the perception of the inner configuration of the agent serves the generation of emotional behaviour, that is, the very capacity of the agent as a whole to adapt its behaviour generation mechanisms to the environment. Note that the emotions themselves are based on those physiological states, used thereby as representations of agent-environment relational situations in virtue of the reliable causal connection. Note also that the content of such emotional states refers to the relevance that the system organization-situation has to the core agent concerns. Conclusion The field of morphological computation has offered important insights on how we should approach the design of embodied intelligent agents. Its ultimate goal is to show that all cognition is grounded in embodiment in one form of another. Its full potential cannot therefore be unveiled until we have a clear view of what is the cognitive and behavioural structure of autonomous agents. Emotion researchers have focussed largely in understanding cognition, emotion and other adaptive phenomena from an architectural point of view: how processes become more complex and meaningful at different levels of processing. From homeostatic mechanisms to full-blown cognition, the phenomenon is seen as embodied. Nevertheless, when this research has been applied to robotics, many details about the interactive nature of behaviour have been lost. In particular, there is little morphological computation involved. The field of morphological computation can nevertheless provide powerful tools for emotion theorists to develop a better understanding of the complex embodied phenomena we can see in humans and animals. In particular, we have advocated for a morphofunctional approach: systems that can change the functionality of their morphology. Emotion, we claim, emerges from the control of the qualitative aspects of the dynamics 49 Deep into Morphology: Emotions and Functional Structure of interaction, through morphofunctional reconfiguration. If successful, this would explain embodied appraisal, a phenomenon of morphofunctional computation that goes beyond basic levels of processing. In short, the control of morphofunctionality relies on feedback variables that come from the body. The situations that would require a change in morphofunctional configuration are exactly those that we call emotional, because they matter and demand a change in the relationship with the environment. Therefore, the link between internal states and relational themes should be grounded in morphofunctional control. We have argued that emotion and morphological computation are thus two very related notions, which rather than competing for attention, should see a greater influence on each other. The potential for cross-fertilization is enormous, and we believe there is great theoretical value in contrasting the two and seeing the resulting picture. We hope this chapter has shown at least in part the ample potential scope for development in this area. Acknowledgements This work has been funded by FP7-PEOPLE-COFUND-2008, through the grant agreement UNITE 246565 between the Technical University of Madrid and the European Commission. We also acknowledge the support of the European Commission through grant HUMANOBS - Humanoids that Learn Socio-Communicative Skills by Observation. Bibliography [1] O. Avila-Garcia and L. Cañamero. Using hormonal feedback to modulate action selection in a competitive scenario. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, and J.-A. Meyer, editors, From Animals to Animats 8: Proc. 8th Intl. Conf. on Simulation of Adaptive Behavior (SAB'04), pages 243–252, Cambridge, USA, 2004. MIT Press. [2] L. Cañamero. Emotion understanding from the perspective of autonomous robots research. Neural networks, 18(4):445–455, 2005. [3] A. R. Damasio. Descartes' Error: Emotion, Reason and the Human Brain. Pan Macmillan, 1994. [4] A. R. Damasio. Looking for Spinoza: Joy, Sorrow and the Feeling Brain. Harcourt, Orlando, Florida, USA, 2003. [5] F. I. Dretske. Knowledge and the flow of information. MIT Press, Cambridge, Mass., 1981. ISBN 0262040638. [6] F. I. Dretske. Explaining Behavior: Reasons in a World of Causes. MIT Press, Cambridge, Mass, 1988. 50 Deep into Morphology: Emotions and Functional Structure [7] C. Edwards, E. Fossas-Colet, and L. Fridman. Advances in Variable Structure and Sliding Mode Control, volume 334 of Lecture Notes in Control and Information Sciences. Springer-Verlag, Berlin, 2006. [8] J. A. Fodor. A theory of content. In A Theory of Content and Other Essays. MIT Press, Cambridge, MA, 1990. [9] N. H. Frijda. The emotions. Cambridge University Press, 1986. [10] N. H. Frijda. The psychologists' point of view. Handbook of emotions, 2:59–74, 2000. [11] P. E. Griffiths and A. Scarantino. Emotions in the wild: the situated perspective on emotion. In P. Robbins and M. Aydede, editors, Cambridge handbook of situated cognition, pages 437–453. Cambridge University Press, Cambridge, 2009. [12] H. Hauser, A. J. Ijspeert, R. M. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological cybernetics, 105:355–370, 2011. [13] C. Herrera-Perez and R. Sanz. Emotion as morphofunctionality. Artificial Life, 19 (1), 2012. [14] C. Herrera-Perez, G. Sanchez-Escribano, and R. Sanz. The morphofunctional approach to emotion modelling in robotics. Adaptive Behavior, 20(5):388–404, 2012. [15] W. James. What is an emotion? Mind, 9:188–205, 1884. [16] R. S. Lazarus. Emotion and adaptation. Oxford University Press, USA, 1991. [17] A. Montebelli, C. Herrera, and T. Ziemke. On cognition as dynamical coupling: An analysis of behavioral attractor dynamics. Adaptive Behavior, 16(2-3):182–195, 2008. [18] OMG. OMG Systems Modeling Language (SysML), Version 1.3. Object Management Group, June 2012. [19] A. Ortony, D. A. Norman, and W. Revelle. Affect and protoaffect in effective functioning. In J. M. Fellous and M. A. Arbib, editors, Who Needs Emotions? The Brain Meets the Machine. Oxford University Press, New York, 2005. [20] J. Panksepp. Affective consciousness: Core emotional feelings in animals and humans. Consciousness and cognition, 14(1):30–80, 2005. [21] D. Parisi and G. Petrosino. Robots that have emotions. Adaptive Behavior, 18(6): 453–469, 2010. [22] R. Pfeifer, F. Iida, and G. Gómez. Morphological computation for adaptive behavior and cognition. In International Congress Series, volume 1291, pages 22–29. Elsevier, 2006. 51 Deep into Morphology: Emotions and Functional Structure [23] R. Pfeifer, M. Lungarella, , and F. Iida. Self-organization, embodiment, and biologically inspired robotics. Science, 318:1088–1093, November 2007. [24] J. J. Prinz. Embodied emotions. In R. C. Solomon, editor, Thinking about Feeling: Contemporary Philosophers on the Emotions, pages 44–59. Oxford University Press, Oxford, New York, 2004. [25] J. J. Prinz. Gut Reactions: A Perceptual Theory of Emotion. Oxford University Press, USA, 2006. ISBN 0195309367. [26] R. Sanz, I. López, and J. Bermejo-Alonso. A rationale and vision for machine consciousness in complex controllers. In A. Chella and R. Manzotti, editors, Artificial Consciousness. Imprint Academic, 2007. [27] A. Sloman. How many separately evolved emotional beasties live within us. In P. P. Robert Trappl and S. Payr, editors, Emotions in Humans and Artifacts, pages 35–114. MIT Press, 2002. [28] T. Ziemke. On the role of emotion in biological and robotic autonomy. Biosystems, 91(2):401–408, 2008. [29] T. Ziemke and R. Lowe. On the role of emotion in embodied cognitive architectures: From organisms to robots. Cognitive Computation, 1(1):104–117, March 2009. 52 Abstract: In this short text we argue that a thorough understanding of the principles underlying the simple act of pushing a small child on a swing might shed light on a wide range of phenomena from brain planning of motion to regulation processes in the cell. Those principles can be summarized by what we call "morphological computation." Zen, Robotics and the Art of Pushing Swings Walking, Swimming and Flying After more than 50 years of Artificial Intelligence (AI) and Robotics research many machines capable of simulating, to a certain extent, animal behaviors like walking, swimming and flying have been proposed. Those machines are still quite far from the level of robustness, flexibility and adaptivity of their natural counterparts. Is this distance in term of performances and capabilities purely quantitative or is there some qualitative difference in the way robots are designed and the organizational principles emerged from natural evolution of animals? The awareness of these gaps pushes us to look to what has evolved in nature with the aim of borrowing "designs" and "solutions". As a consequence, bio-inspiration is a popular trend in robotics. Which kind of inspiration should we take from the biological world when designing "intelligent" artefacts? We should, probably, chase a "deep" one, we should aim to design systems by exploiting the underlying principles leading to the organization of processes and structures that we observe in natural intelligent agents. Let us consider popular humanoid designs such as those of Sony's QRIO, Honda's Asimo and the likes. Although they have roughly human appearances they are based on a completely different approach: pieces of steel and plastic (more steel than plastic, to increase inertia) moved by algorithmic controllers running on a network of microprocessors. If we look at how they move we perceive it as "different" from the way people or animals move. The design paradigm of modern robots and androids has not much changed from the first Greek automatons from Csetibius and Heron: a mechanical system (made of variously connected rigid bodies) controlled by a state machine (a clock machine in ancient times, an algorithm running on one or more microprocessors today). However, we may think that we can outperform natural systems. We may observe that wheeled locomotion and flying by jets are preliminary not bio-inspired (although examples of "wheels" can be found in special behaviors of bacteria and other animals or in the "desert rose" while jet-propulsion is used by some octopus for swimming). These two later examples are sometimes used as arguments that we should not look at nature for inspiration: after all it seems apparent that wheels are more energy efficient than legs and jet airliners go faster than any bird. This is not completely true. Consider, for example, locomotion. Wheels are useful if you have to move on planar rigid surfaces with appropriate friction, however, they are not very useful in a forest and they are difficult to use on a sandy beach or a slippery street. In an unstructured natural environment legs are better. Interestingly a fully actuated humanoid walker requires 20-30 times the power required by a human. Passive walkers like the Cornell Ranger [1], and previous designs like the passive walkers proposed by MIT, Cornell and Delft more than 10 years ago have power consumptions of the same order of magnitude of humans and no, or only a very simple, control system (in the case of the MIT biped a reinforcement learning schema adapting the impulse given once for every gait). This is made possible by the fact that part of the control is offloaded to the natural body dynamics, see [13, 4, 6]. Regarding swimming and flying, we are in a similar situation. We have submarine-like underwater robot systems and more energetically and computationally frugal swimming 54 Zen, Robotics and the Art of Pushing Swings systems like fishes, like Wanda, depicted in Figure 1 [28], and fixed wing jet propelled drones and bio-inspired flying or jumping machines [12, 25]. The level of maneuverability of a bird is still much higher than what is achievable by an airplane, even with a very skilled human pilot in the control loop. Figure 1: The robot fish Wanda. This figure represents a robot fish developed at the AILab of the University of Zurich. It can swim by making its deformable body oscillate. It is considerably more simple than a submarine-like robot as it needs only a single motor to move. It is an early example of underwater robot exploiting morphological computation: the control is simpler thanks to its morphology, as the natural dynamics leads to the desired behavior.(Courtesy of the authors taken from [28]) In all these cases, passive walkers, Wanda, birds, we see two main organizational principles: • exploiting the natural system dynamics • offloading (at least part of) the computational burden of the control to the system dynamics If you look at how a passive walker is designed you will see that the morphology (legs' length, mass distribution, etc.) drives the possible limit cycles of the system, which result in corresponding gaits. Yet, "if you want to change speed with the Cornell Ranger you have to change the robot". If we consider an ideal passive walker where we are allowed to change the morphology depending on the desired gaits (and speed) we can have the same performances in term of maneuverability (for example we can change speed or make turns) at a small fraction of energy and computational cost. Hence, we need a deformable structure, a "soft robot". Soft robotics lead to serious challenges from the control engineering, and material science, standpoint, but there are fundamental reasons to believe it could enable dramatic improvements in the robot capabilities. 55 Zen, Robotics and the Art of Pushing Swings Figure 2: Different size passive walkers. As shown in [13] the gait (and average speed) depends on the morphology. The different simplified walkers repesented in this picture will move with different speeds. You need to change the lenghts of legs, i.e. a deformable structure. Emerging Orchestration of Behaviors in Networked Embodied Agents There is a famous horrific scientific experiment movie that shows a cat walking fluently on a treadmill. The horror comes from the fact the cat is dead. The lesson learned is that a cat, dead or alive, does not need much control and energy to walk. The control and the energy involved in the dead cat experiment are those provided by the treadmill. Living cats might be seen as a (much) more elaborated version of the variant morphology passive walker quoted above. Considerations like this naturally lead to see the problem of optimal, in terms of energy and computation, control of (passive) walking systems (actually of any moving system) as an "orchestration" problem. For example, if we want to change the speed of a deformable passive biped the actuators will change the morphology of the system instead of acting directly on the feet motion law. Feet motion originates from the morphology's interaction with the environment, from the natural dynamics of the physical embodied systems, as explained in Figures 2 and 3. In nature intelligent behaviors emerge from loosely coupled networks of embodied agents. The prevailing organizational pattern for behavior generation in the animal world seems to be based on orchestration, self-organization, Bayesian prediction and emergence, see [7], in embodied, and situated, agents. The understanding of the trade-offs between morphology, dynamics and information in this context is the core issue of "morphological computation". 56 Zen, Robotics and the Art of Pushing Swings Figure 3: How to change the length of a leg.This picture shows that you can actually change the length of the legs with a trick: you need a segmented leg, for example, one including a foot. The orchestration control will change the average speed of the walker by changing the angle ✓ formed by the foot with the leg in accordance with the desired average speed. As a consequence the equivalent lenght of the leg l, depending on l1, lf and ✓, and the average period of the limit cycle T ⇤gait will also change, affecting the time requested for a gait and the resulting speed of the walker. This is remarkably less complicated than imposing a given deterministic trajectory to the foot by controlling the hip joint with an actuator. The usage of fixed-structure rigid bodies limits the range of achievable speeds for a given walker. Natural "controllers" are not only "orchestrating" ones, but also self organized and exploiting emerging dynamical behaviors. In an open-ended world you cannot rely on a predesigned control strategy you need that the control strategy self organize as a response to change. These observations seem really compelling if you look carefully to some aspects of our common experience. Zen and the Art of Pushing a Swing Let us consider a woman pushing a child on a swing. You see at work at the same time all the main principles of cognitive control you find in nature. There is little energy usage, as the pendulum movement of the swing is a consequence of Newton's inertia law. Only the energy loss due to friction and the aerodynamic drag needs to be restored. You need prediction capabilities, as the movement of the child influences the aerodynamic resistance, and the position and attitude of the pushing surface at the moment of the woman push. The control has to be self organized as it needs to cope with the chaotic behavior of the child, and it is emergent as it come out from the chaotic oscillatory, or better pseudo-oscillatory, behavior of the massive 57 Zen, Robotics and the Art of Pushing Swings network of networks inside the brains of the woman and the child. Moreover, the gaze, movements and cognitive and emotional processes of the woman and of the child need to be synchronized to allow efficient and effective prediction of the swing movement. Like the archer of the famous book, "Zen in the art of archery" [14], the woman need to clean the intellectual superstructures and concentrate, to be open to her inner world, to her "Zen." In our perspective, she has to let suitable sensory motor coordination schemes to emerge naturally. It is, by the way, extremely interesting from a traditional Western cultural perspective, what Buddhism labels as "embodiment". The Buddhists do not see an abstract entity into a separated body, but they see the body as the perception and actuation part of the "ego." You have a different "umwelt" [3, 26], if you are embodied as a monkey, a human or as an ant. In the simple behavior of pushing a swing we actually exploit all the main principles that allow us to survive in nature. These mechanisms are not confined to the more recent products of evolution, but they are governing the more simple examples of living beings like the cells. If you think about the critical importance of morphology, dynamics (for example of DNA, RNA, proteins), stochasticity and emergence, in the processing of information in the cell, cell regulation itself might be regarded as the emergent, self-organizing, predictive orchestration of a huge network of networks of loosely coupled embodied agents. We may speculate that life, natural cognition, and consciousness, themselves are kinds of a spontaneous dance of the matter. Open Challenges, Possible Solutions and Needed Breakthroughs Many people share the vision that we have outlined above. The idea that natural cognitive behaviors in natural intelligent systems, spanning from bacteria to humans, spring from the emergent, orchestrating, predictive control of self-organized loosely coupled networks of embodied and situated agents seems convincing from a qualitative standpoint,and clear and deep arguments have been provided, mainly by Pfeifer [20, 21, 19]. Why cannot we, then, exploit this approach to build artificial intelligent systems of unprecedented robustness, flexibility and adaptivity? We have, in my opinion, two main unsolved problems: • we lack a quantitative framework theory explaining and modeling those phenomena • we do not have suitable material technologies to implement the design concepts in artificial systems In turn such quantitative models and the possibility of testing hypotheses on natural systems with artificial models, (the "understanding by building" or "synthetic" methodology) would allow dramatically deeper insights into the natural cognitive processes. The ability to model the emergence of self-organized behaviors, from sensory-motor coordination to more complex "purposeful" tasks in deformable ("soft") materials, characterized by a fine distribution of sensing and actuation is also needed to guide the 58 Zen, Robotics and the Art of Pushing Swings development of new composite materials that we need to emulate natural bodies and to reverse engineer them. Preliminary work [2, 11, 17], on information driven self-organization (IDSO) on stochastic kinematics [8, 9, 27] and merging the two [5, 6], suggest that the task, certainly tough, might not be impossible. Moreover methods from differential geometry might help to find abstract, but powerful solutions. The "embodiment" of the agents shapes its possible interactions with the environment, and the consequent "limitations" can mathematically be represented by Lie Groups. The experiments reported in [16] show that the maximization of suitable informational metrics (inspired by Shannon entropy) might be important to model the emergence of cognitive processes in natural and artificial intelligent systems. This is usually referred to as Information Driven Self Organization. Snakebot by Tanev [23, 24], a system designed according to IDSO principles, or the hexapod walking model proposed by Cruse [10], where walking behavior emerges, without any central controller, through the interaction of the embodied system with the environment are examples of the kind of models we should develop. It has also be shown that the interaction of the embodied agent with the environment shapes the basic perception of the agent, see [22]. Figure 4: The Snakebot. The Snakebot's simulated crawling is an example of emerging behavior based on Information Driven Self Organization. The snake is modeled by a series of loosely coupled rigid balls. The maximization of the predictive information between the parts of the system leads to a crawling behavior very similar to that of the crotalus.(Courtesy of the authors -taken from [24]) The maximization of suitable Shannon-information based functionals to generate selforganized behaviors produces intuitively convincing results. Yet the direct application of these methods leads to excessively onerous computations. It is possible that "embodying" the methods by considering the representation of the motion structure of the physical body interacting with the environment would drastically simplify the problem. Methods useful to do that are proposed in [8, 9, 5, 18, 15] and in upcoming publications by the author. 59 Zen, Robotics and the Art of Pushing Swings Discussion and Future Work Uncovering the principles of morphological computation and translating them into quantitative models are a decisive step towards a new understanding of intelligent behaviors in natural systems and towards a new design approach to robotic systems. These new principles have at the same time tremendous importance from the scientific standpoint and a dramatic potential impact on the design of embedded systems (from cars, to smart homes to logistics and supply chain networks). They are a cornerstone of a new science, and technology, of physical cognitive systems. The qualitative models based on these principles are well understood and the mathematical tools are likely to be available. Progress in material science is quick and impressive and may benefit by new models of materials where "intelligence" is finely distributed. We have in front of us serious challenges, but there are many hints that we can cope with them. During the Renaissance, Leonardo da Vinci conceived and dreamed of different types of mechanical flight, fixed wing, helicopter and winged bird-like flight. The latest has still to be achieved. Would it be achievable with a better understanding of the relation between information processing and body dynamics, and new materials? A deep and quantitatively grounded understanding of the trade-offs (between morphology, dynamics and information) and the control mechanisms involved by the simple act of pushing a swing might help us to develop a radically new robotics technology built around the principles of a new science of physical cognitive systems, and, at the same time, allow us to gain much deeper insights on basic natural processes spanning from internal cell regulation to human brain motion planning. Bibliography [1] Cornell Ranger website, July 2012. [2] N. Ay, N. Bertschinger, R. Der, F. Güttler, and E. Olbrich. Predictive information and explorative behavior of autonomous robots. The European Physical Journal B-Condensed Matter and Complex Systems, 63(3):329–339, 2008. [3] A. Berthoz. Neurobiology of "Umwelt" How Living Beings Perceive the World. Springer, 2009. [4] F. Bonsignorio. Preliminary considerations for a quantitative theory of networked embodied intelligence. In M. L. et al., editor, 50 Years of AI, Festschrift, pages 112–123. Springer, Berlin – Heidelberg, 2007. [5] F. Bonsignorio. On the stochastic stability and observability of controlled serial kinematic chains. ASME 2010 10th Biennial Conference on Engineering Systems Design and Analysis ESDA2010, Istanbul, 2010. ASME. [6] F. Bonsignorio. Quantifying the evolutionary self-structuring of embodied cognitive networks. Artificial Life, 19(2):267–289, 2013. 60 Zen, Robotics and the Art of Pushing Swings [7] F. Bonsignorio. Steps to a cyber-physical model of networked embodied anticipatory behavior. In G. P. et al., editor, ABIALS 2008, pages 77–94. Springer, Berlin – Heidelberg, 2009. [8] G. Chirikjian. Information theory on Lie-groups and mobile robotics applications. ICRA 2010, Anchorage, 2010. IEEE. [9] G. Chirikjian. Stochastic models, Information Theory, and Lie groups, Vol. 2. Birkhauser, Boston, 2011. [10] H. Cruse. Neural Networks as Cybernetic Systems. Thieme, Stuttgart, 1996. [11] R. Der, G. Martius, and F. Hesse. Let it roll – emerging sensorimotor coordination in a spherical robot. Artificial Life X, pages 192–198, Boston, 2006. MIT Press. [12] D. Floreano, J.-C. Zufferey, M. V. Srinivasan, and C. Ellington, editors. Flying Insects and Robots. Springer, 2009. [13] M. Garcia, A. Chatterjee, A. Ruina, M. Coleman, et al. The simplest walking model: Stability, complexity, and scaling. J Biomech Eng Trans ASME, 120(2): 281–288, 1998. [14] E. Herrigel. Zen in the art of archery. 1948. [15] I. Kolar, P. Michor, and J. Slovak. Natural Operations in Differential Geometry. Springer, Berlin – Heidelberg, 1994. [16] M. Lungarella and O. Sporns. Mapping information flow in sensorimotor network. PLOS Computational Biology, 2(10):1301–1312, 2006. [17] L. Olsson, C. Nehaiv, and D. Polani. Information trade-offs and the evolution of sensory layouts. Artificial Life IX, 2004. [18] P. Olver. Applications of Lie Groups to Differential Equations. Springer, Berlin – Heidelberg, 2000. [19] R. Pfeifer and J. Bongard. How the body shapes the way we think: A new view of intelligence. Bradford Books, Boston, 2006. [20] R. Pfeifer and M. Maris. Cheap designs: exploiting the dynamics of the systemenvironment interaction. three case studies on navigation. Conference on prerational intelligence, phenomenology of complexity emerging in systems of agents interacting using simple rules, pages 81–91. Springer, 1993. [21] R. Pfeifer and C. Scheier. Understanding intelligence. MIT Press, Boston, 1999. [22] D. Philipona, J. O'Regan, J. Nadal, and O.-M. Coenen. Perception of the structure of the physical world using unknown multimodal sensors and effectors. Advances in Neural Information Processing Systems 16. MIT Press, 2004. 61 Zen, Robotics and the Art of Pushing Swings [23] M. Prokopenko, V. Gerasimov, and I. Tanev. Evolving spatiotemporal coordination in a modular robotic system. From Animals to Animats 9: 9th International Conference on the Simulation of Adaptive Behavior. Springer, 2006. [24] I. Tanev, T. Ray, and A. Buller. Automated evolutionary design, robustness, and adaptation of sidewinding locomotion of a simulated snake-like robot. IEEE Transactions on Robotics, 21(4):632–645, 2005. [25] R. Tedrake, Z. Jackowski, R. Cory, J. Roberts, and W. Hoburg. Learning to fly like a bird. Massachusetts Institute of Technology Computer Science and Artificial Intelligence Lab, Tech. Rep, 2006. [26] J. v. Uexkull. A Stroll Through the Worlds of Animals and Men: A Picture Book of Invisible Worlds, Instinctive Behavior: The Development of a Modern Concept. International Universities Press, Inc, 1957. [27] Y. Wang and G. Chirikjian. Error propagation on the euclidean group with applications to manipulator kinematics. IEEE Transactions on Robotics, 22(4):591–602, 2006. [28] M. Ziegler, F. Iida, and R. Pfeifer. "Cheap" underwater locomotion: Roles of morphological properties and behavioural diversity. Proceedings of Climbing and Walking Robots, 2006. 62 Abstract: In the natural world computation is a matter of survival. Organisms must utilise their own resources in the most efficient way to exploit their environment for food, shelter and reproduction opportunities. Adaptation is achieved at both the individual and population level within a changing environment. For the giant single celled organism slime mould Physarum polycephalum this adaptation is literal: changing its entire body plan during growth, movement, foraging, feeding, and hazard avoidance. Slime mould is remarkable because, despite possessing no nervous system, it has been shown to perform remarkable feats of computation. Slime mould may thus be considered as a living material form of morphological computation. We explore how the concept of material computation by morphological adaptation can be extracted from the behaviour of slime mould to develop distributed multi-agent collectives with emergent, quasi-material behaviour. The natural pattern formation and network minimisation of this virtual material may be influenced by the application of environmental stimuli to perform useful computation. We give a brief overview of the approach and demonstrate how the idea of virtual material adaptation can be used for spatially represented unconventional computation and distributed robotics tasks. We conclude by examining the possible future roles and challenges facing material based morphological computation. A Virtual Material Approach to Morphological Computation Introduction – Morphological Computation and Natural Systems Morphological computation seeks to directly utilise the embodied properties of synthetic systems and their interactions with the environment in which they exist [38]. The motivations for this approach are twofold. The first is based on necessity caused by the increasing complexity of computing systems. Traditional approaches in integrating separate complex systems (for instance, sensors, actuators and control systems in robotics) result in complex components with little redundancy or fault tolerance. Typically it is difficult to control such systems as the size of the system increases. The second motivation has more positive origins. Can we utilise the natural, material properties of synthetic systems to improve the performance (or minimise the difficulties faced) when constructing synthetic systems? Furthermore, can we extend the properties of embodiment to include the environment and take advantages of the processes which occur naturally within it? We can gain some help in investigating approaches to morphological computation by examining the natural world. Nature inspired computing and robotics takes inspiration from systems which are composed of a great many relatively simple parts and which are embedded within complex environments. For example, swarm computation approaches seek to elucidate the sensory mechanisms and individual interactions which generate the complexity patterning and movement seen at very different scales in natural systems including car traffic dynamics [16], human walking patterns [17], flocking and schooling [42], collective insect movement [10], and bacterial patterning [33]. In all these examples there is a population of entities in space, coupled by sensory information about their environment. The collective morphology of the group is generated from the local interactions and movement of individual members of the population. These interactions generate complex, self-organised and emergent behaviour. The resulting morphological patterns of the population show adaptation to the environment, for example avoidance of obstacles, detection of prey, and avoidance of predators. The tenets of emergent behaviour (simple, local interactions with self-organisation) may also be exhibited in simpler systems which straddle the boundary of non-living physical materials and living organisms including those acting as biological fibres and membranes [60], lipid self assembly in terms of networks [30], pseudopodium-like membrane extension [31] and even those exhibiting simple chemotaxis responses [29]. Some engineering and biological insights have already been gained by studying the structure and function of what might be termed "semi-biological" materials and the complex behaviour seen in such minimal examples raises questions about the lower bounds necessary for the emergence of apparently intelligent behaviour. It has also been suggested that simple material behaviour, as opposed to complex living systems, may provide a rich vein of potential computational resources to be explored [48]. 64 A Virtual Material Approach to Morphological Computation Classical and Unconventional Computation The above examples of collective morphological adaptation in biological and semi-biological systems suggest a literal form of morphological computation, driven by dynamic patterning and responding to a complex environment. Such computation is very different from classical computation and a brief overview is required to delineate the differences. The current dominant model of computing, referred to as classical computation, is based on silicon embodiments of Turing's notion of a Universal Machine. The success of the approach is due to the concept of the stored program computer, whose behaviour is governed by a sequence of simple logical and arithmetic symbolic instructions. Problem data, and their solutions, must be transformed into a representation which allows the solution by the machine instructions. The individual instructions are very simple indeed but, when executed, (subject to iteration and conditional branching to different parts of the instruction stream) they transform the input data to yield output data containing the problem solution. Both instructions (program) and data may be represented in the same medium and the behaviour of the computer may be changed by simply changing the contents of the instructions and data storage. Thus a single mechanism may be driven by its program and data to "become" any other machine, as long as the desired behaviour is known and completely specifiable. The success of the classical approach is partly due to this flexibility of its architecture and partly due to the vast speed at which modern digital computing devices operate. Unconventional computation, or non-classical computation, is an approach whereby the natural properties and processes of physical or living materials are harnessed to provide useful computational functions. The motivation for the study of unconventional computation is threefold. Firstly, many natural systems exhibit properties which are not found in classical computing devices, such as being composed of simple and plentiful components, having redundant parts (i.e. not being dependent on highly complex units), and showing resilient or "fault tolerant" behaviour. Secondly, non-classical computation is often observed in systems which show emergent behaviour. Although the definition of emergence is difficult to define precisely and the subject of debate [43, 8] it may be summarised as being novel behaviour which emerges from the interactions between simple component parts, and which - critically - cannot be described in terms of the lower level component interactions. Emergent behaviour is characterised by systems with many simple, local interactions and which display self-organisation, i.e. the spontaneous appearance of complexity or order from low-level interactions. Many of the attractive features of non-classical computing devices (redundancy, fault tolerance) are thought to be based on the mechanisms of self-organisation and emergence, and the study of these properties is useful not only from a computational perspective, but also from a biological viewpoint – since much of the complexity in living systems appears to be built upon these principles. The third reason for interest in non-classical computing is because, for a number of applications at least, utilising the natural properties of physical systems for computation is a much more efficient means of computation. Non-classical computation can take advantage of massively-parallel computation within the computing medium. For 65 A Virtual Material Approach to Morphological Computation example, in the chemical approximation of Voronoi diagrams [12] the substrate is composed of thousands of "micro-volumes" – individual regions of the substrate through which information is propagated between local neighbours and through which computation is approximated (by the presence or absence of precipitate formation). In addition to parallelism, the sheer speed of operation within the computing substrate may be advantageous. In the approach of Reyes et al. [41], potential path choices on a substrate comprised of gas-discharge plasma within an etched microfluidic map, were evaluated almost instantaneously to yield a visual representation of the shortest path. Non-classical computing is not without its disadvantages, some of which are mainly theoretical. It may be difficult to represent certain computational problems in a way which can harness the physical behaviour of the non-classical substrate. Non-classical substrates may be "one-shot" computing entities, not capable of reconfiguration or re-use (although some authors contend that this is an advantage, [59]). Certain problems which are suited to symbolic or logical transformation may be difficult to translate into spatial and propagative formats. This difficulty manifests itself in terms of how to represent data input and also how to "read" the problem solution. This problem is exacerbated when considering how to interface non-classical computers to traditional computers to exploit the latter's storage, archival and search abilities. The issue of when to stop the computation also arises, since natural systems tend not to halt when a problem is solved, but often tend to adopt a dynamic equilibrium among different states. More practical problems also arise from non-classical approaches. They may be made from exotic materials or at challenging scales (such as DNA [15], enzymes [39], plasmas [41, 58]) and as such may be expensive and difficult to fabricate. Alternately they may simply be impractical, relatively slow, or ill suited to performing certain operations, which would be more efficiently computed by classical approaches. One alternative to utilising the actual physical or living system to perform non-classical computation is to try to abstract the most salient features of the system in question and encode these behaviours within classical algorithms. A notable example is evolutionary computation, where population variation mediated by mutation and recombination is incorporated into a population of "chromosomes" representing program parameters [34] or the programs themselves [28]. Even more relevant to this article is the example of Ant Colony Optimisation, a family of meta-heuristic approaches where the phenomena of pheromone sensing and deposition by ant colonies is abstracted and used to enforce a cost to different paths in combinatorial optimisation problems [13]. In all of these examples, however, the complex spatial and physical interactions of the source systems are lost during the abstraction to a classical encoding. The abstracted models are thus biased towards the assumptions as to which physical features are responsible for the complex computation and risks losing a rich potential well of computational power freely available in the physical system. 66 A Virtual Material Approach to Morphological Computation Morphological Computation as Performed by Slime Mould An ideal hypothetical candidate for a morphological computation medium would be a material which is capable of the complex sensory integration, movement and adaptation of a living organism, yet which is also composed of relatively simple components that are amenable to understanding and control. The Myxomycete organism, the true slime mould Physarum polycephalum, may be a suitable candidate medium which meets both criteria; i.e. it exhibits complex behaviour, but which is composed of relatively simple materials. P. polycephalum is an attractive candidate medium for morphological computation because it is a literal example of morphological adaptation. The slime mould is a giant single-celled organism which can usually be seen with the naked eye (see http://en.wikipedia.org/wiki/Physarum_polycephalum for a brief overview or [47] for a more detailed description). During the plasmodium stage of its life cycle it adapts its body plan in response to a range of environmental stimuli (nutrient attractants, repellents, hazards) during its growth, foraging and nutrient consumption. The plasmodium is composed of a transport network of protoplasmic tubes which spontaneously exhibit contractile activity which is harnessed used in the pumping and distribution of nutrients. The organism is remarkable in that its complex behaviour is achieved without any specialised nervous tissue. Control of its behaviour is distributed throughout the simple material comprising the cell and the cell can survive damage, excision or even fusion with another cell. The plasmodium of slime mould is amorphous in shape and ranges from the microscopic scale to up to many square metres in size. It is a giant single-celled syncytium formed by repeated nuclear division, comprised of a sponge-like actomyosin complex cooccurring in two physical phases. The gel phase is a dense matrix subject to spontaneous contraction and relaxation, under the influence of changing concentrations of intracellular chemicals. The protoplasmic sol phase is transported through the plasmodium by the force generated by the oscillatory contractions within the gel matrix. Protoplasmic flux, and thus the behaviour of the organism, is affected by changes in pressure, temperature, space availability, chemo-attractant stimuli and illumination [11], [14], [27], [35], [37], [50], [56]. The P. polycephalum plasmodium can thus be regarded as a complex functional material capable of both sensory and motor behaviour. Indeed P. polycephalum has been described as a membrane bound reaction-diffusion system in reference to both the complex interactions within the plasmodium and the rich computational potential afforded by its material properties [7]. The study of the computational potential of the P. polycephalum plasmodium was initiated by Nakagaki et al. [36] who found that the plasmodium could solve simple maze puzzles. This research has been extended and the plasmodium has demonstrated its performance in, for example, path planning and plane division problems [45],[46], spanning trees and proximity graphs [2], [1], simple memory effects [44], the implementation of individual logic gates [53] and P. polycephalum inspired models of simple adding circuits [24]. From a robotics perspective it was shown that by its adaptation to changing conditions within its environment, the plasmodium may be considered as a prototype micromechanical manipulation system, capable of simple and programmable robotic actions 67 A Virtual Material Approach to Morphological Computation including the manipulation (pushing and pulling) of small scale objects [5], transport and mixing of substances [3] and as a guidance mechanism in a biological-mechanical hybrid approach where the response of the plasmodium to light irradiation was used to provide feedback control to a robotic system [54]. The notion of a functional material utilising protoplasmic streaming was suggested by Ishiguru et al. [18] who noted the potential utility of distributed and emergent control within the material. A P. polycephalum inspired approach to amoeboid robotics was also demonstrated by Umedachi et. al. [57] in which an external ring of coupled oscillators, each connected to passive and tuneable springs was coupled to a fluid filled inner bladder. The compression of the peripheral springs mimicked the gel contractile phase and the flux of sol within the plasmodium was approximated by the coupled transmission of water pressure to inactive (softer) springs, thus deflecting the peripheral shape of the robot. The resulting movement exhibited flexible behaviour and amoeboid movement. Reproducing Slime Mould Behaviour: A Multi-Agent, Virtual Material Approach Although slime mould has enviable computational properties, it also has limitations due to the fact that it is a living organism. Although simple and inexpensive to culture, slime mould is also relatively slow (certainly compared to silicon computing substrates) and must be maintained within strict environmental parameters of temperature, light exposure and humidity. Slime mould may also be relatively unpredictable in its behaviour. Although the unpredictability is useful in certain circumstances it can be a hindrance when repeatable measures of its performance are required. We therefore require a synthetic analogue of slime mould. One technique available is computer modelling, where we attempt to reproduce the complex patterning of slime mould along with the complex interactions it has with its environment. It is important to note, however, that we are not simply trying to extract the features of slime mould for classical algorithms. Such an approach may indeed prove useful for certain tasks, but would not inform us in any way about the distributed emergent behaviour and control of the organism. Instead what we wish to do is construct a virtual material using the same principles (and apparent limitations) of slime mould, i.e. using only simple component parts and local interactions. The aim is to generate collective emergent behaviour utilising self-organisation to yield an embodied form of material computation which can reproduce the wide range of complex patterning and environmental responses seen in slime mould. Guiding Pattern Formation for Computation An approach we devised was given in [19] in which we introduced a large population of simple components, mobile agents, (a single agent is shown in Figure 1) whose behaviour was coupled via a diffusive chemo-attractant lattice. Agents sensed the concentration of a hypothetical "chemical" in the lattice, oriented themselves towards the locally strongest 68 A Virtual Material Approach to Morphological Computation source and deposited the same chemical during forward movement. The collective movement trails spontaneously formed emergent transport networks which underwent complex evolution, exhibiting minimisation and cohesion effects under a range of sensory parameter settings. The collective behaves as a virtual material demonstrating characteristic network evolution motifs and minimisation phenomena seen in soap film evolution (for example the formation of Plateau angles, T1 and T2 relaxation processes and adherence to von Neumann's law [23]). A full exploration of the dynamical patterns were explored in [20] which found that the population could reproduce a wide range of Turing-type reaction-diffusion patterning (Figure 2). (a) (b) Figure 1: Base agent particle morphology and sensory stage algorithm. (a) Illustration of single agent, showing location "C", offset sensors "FL","F","FR", Sensor Angle "SA" and Sensor Offset "SO", (b) simplified sensory algorithm. Nutrients may be represented by projecting chemo-attractants into the lattice at fixed sites and the network evolution is constrained by the distribution of nutrients. Network evolution was affected by nutrient distribution and nutrient concentration. An example of network minimisation using the virtual material approach is given in Figure 3 where the network assembles and minimises around two point sources of attractants. Figure 4, illustrates the adaptive morphology of the population, initialised as a large sheet-like mass, as it adapts its shape to the spatial configuration of nutrients. It was subsequently found that the emergent transport networks reproduced the connectivity of slime mould by approximating networks in the Toussaint hierarchy of proximity graphs [22], as originally demonstrated in [1]. Building upon the finding that network configuration in the model was influenced by global nutrient concentration we investigated whether the material could be subject to local and dynamic control of patterning in real-time, using a feedback mechanism based upon adjusting nutrient concentration in response to current network connectivity [21]. It was found that real-time control of spatial configuration was possible in relatively simple examples, and characteristic high-level motifs in network evolution emerged as a result of the feedback process that helped the network migrate between different configurations. The virtual material could be used as a morphological computation mechanism to find 69 A Virtual Material Approach to Morphological Computation Figure 2: Parametric mapping of agent sensory parameters yields complex and dynamical reaction-diffusion patterning. Figure 3: Pattern formation and evolution under the influence of nutrient stimuli. Initial image (top-left) shows: pre-pattern cues (dark spots), initial agent positions (small grey flecks) and boundary of the environment (uniform grey), Remaining images (left to right): Self-assembly and evolution of agent transport network as it becomes "snagged" by the attractant stimuli. Minimisation of the network continues until the shortest path between the stimuli remains. 70 A Virtual Material Approach to Morphological Computation Figure 4: Approximation of Steiner tree by shrinkage of virtual plasmodium. Population initialised as a large sheet covering square arrays of regularly placed simulated nutrients. Shrinkage of sheet over time yields approximation of Steiner minimum tree in 4x4 (top), 5x5 (middle), and 7x7 node arrays. solutions to small instances of the Travelling Salesman Problem (TSP). More recently, it has also been demonstrated that morphological adaptation of the virtual material (shrinking a solid "blob" of the material) can passively approximate good solutions to the TSP [26]. From Material Oscillatory Phenomena to Collective Transport and Amoeboid Movement The virtual material approach captures the idealised pattern formation and adaptation of slime mould in response to environmental conditions. However the other fascinating property of slime mould is how it harnesses the spontaneous oscillatory contractions within its cell to effect the changes in morphology, achieving these feats using identical components and under distributed control. Slime mould uses the motive force provided by the oscillatory contractions during its growth, foraging, nutrient consumption, network adaptation and escape from hazardous conditions and an understanding of how this may be reproduced would be of benefit, particularly to the field of soft-bodied robotics. By adding transient resistance to the movement of particles in the model we were able to reproduce the spatio-temporal oscillation patterns observed by Takagi and Ueda [49] in P. polycephalum, and reproduce the gradual synchronisation of oscillatory activity of the plasmodium [51], [52]. The presence and timing of oscillatory activity in real Physarum plasmodium and the multi-agent model was used to indirectly predict the current state, position and network configuration of transport networks [6]. Exploration of oscillatory patterning in arenas completely occupied by large "sheets" 71 A Virtual Material Approach to Morphological Computation of virtual plasmodium found the emergence of travelling waves which underwent competition and entrainment and subsequently were used to generate regular transport patterns [55], which were used to transport simulated objects (Figure 5). (a) (b) Figure 5: Simulating passive transport of substances using information from travelling waves. (a) Spatial representation of snapshot of emergent travelling waves within virtual plasmodium. (b) Computed vector field based upon direction and gradient of travelling waves is used to move passive objects (circular shapes) by cilia-like transport. In smaller collectives ("blobs" of virtual material), the natural cohesion of the collective contained the internal travelling waves to yield collective movement of the blobs. It was found that, as with Physarum plasmodium [4], the virtual material could be controlled using attractants (simulated nutrient projection, see Figure 6) and hazards (exposure to simulated light irradiation). Furthermore the collective could migrate through experimental arenas that were narrow or tortuous and could be split into, and later re-fused from, independently controllable entities [25]. Possible Applications of The Virtual Material Approach in Other Systems Using the multi-agent virtual material approach we have demonstrated that reproducing the wide range of morphological patterning and behaviour seen in slime mould is possible using a very simple method. Can the method be applied to other approaches or modelling tasks? It is notable that particles with identical shapes but opposite behaviour of those considered in this article (where particles are repelled by chemo-attractant) also show very complex pattern formation and evolution [20]. We must also consider the possibility of complex patterning by agents with completely different shapes (e.g. particles with fewer, or more, sensors, arranged in different architectures), suggesting that the simple agent based method of morphological computation may be applicable to modelling other systems, or be used for other applications. Preliminary research in these areas has found that the method is capable of approximating phase separation and interfacial mixing 72 A Virtual Material Approach to Morphological Computation (a) (b) Figure 6: Collective amoeboid movement towards chemo-attractant source. (a) chemo-attractant source is projected into diffusion field (small dark circle), provoking extension of the virtual material at its periphery. The material engulfs the source as emergent travelling waves move toward the source. The material blob re-adopts its original shape when nutrients are exhausted, (b) Illustration of chemo-attraction to the nutrient source (left), Migration of leading particles towards source (middle), Emergence of travelling waves pull the collective, engulfing the source (right). 73 A Virtual Material Approach to Morphological Computation patterns (both using a combination of two opposite agent types). In many materials, altering the environment or properties of the material (for example growing boundaries or shrinking materials in drying conditions) place stresses on materials which respond by adapting their patterns [9]. Preliminary experiments using the multi-agent method reproduce the material response to changing boundaries by forming septate patterns (the formation of boundaries separating regions within a space) or fragmentation into droplets, suggesting a possible application in the modelling of these processes. Most examples used in the approximation of slime mould using the virtual material method use simple pointlike sources of chemo-attractant to attract or anchor the virtual material. More complex chemo-attractant gradients would also be expected to expand the patterning repertoire of the multi-agent approach and preliminary experiments have demonstrated the formation of a range of complex and self-organised phyllotaxis-like spiral patterns. Extending the model into three-dimensional space may also extend the 2D area minimisation capabilities of the modelling approach towards volumetric surface area minimisation problems. Computational Perspectives on Material Approaches to Morphological Computation From a computational perspective the virtual material approach is a spatially represented non-classical (or unconventional) computation system. Both problem specification and output solution are represented by spatial patterns and problems are solved by iterating the natural dynamics of the multi-agent collective until a stable pattern is reached. The collective behaviour of the material is a literal form of morphological computation – the shape of the material is deformed by, and adapts to, the configuration of simulated attractants and hazards in its environment. One of the benefits of studying material approaches to computation using semi-biological materials, whether real or virtual, is that it allows us to examine synthetic methods of generating complex behaviour from simple components without being restricted by the difficulties of using living systems (for example the problems in keeping such systems alive, providing energy and removing waste products). We have found that very simple behaviours - indeed surprisingly simple - can reproduce the specific patterns of growth of slime mould and provide a plausible account for its efficiency in foraging and its complex oscillatory activity. The apparent irony at the use of classical computing devices to generate a virtual material for use in the study of unconventional computing substrates must also be noted. We must also distinguish the difference between computing substrate (whether silicon based or natural materials) and computation architectures (for example iterative sequential instructions on von-Neumann designs or spatially represented propagative computation). It is certainly true that the model runs as a program on a classical PC. However we have explicitly restricted ourselves to using the same limitations as seen in slime mould, i.e. simple component parts and local interactions. The complex adaptive quasi-physical behaviours and distributed control of the virtual plasmodium emerge from these simple computations, and these second-order emergent behaviours - critically - cannot be 74 A Virtual Material Approach to Morphological Computation described in terms of the program instructions which generate them. As long as we restrict our interactions with the material to these second-order emergent behaviours (for example by placement of spatial stimuli or hazards) we can effectively ignore the underlying classical computation which generates the material behaviour. It should also be noted that the same conceptual problem is also faced by other non-classical computing approaches, for example cellular automata and lattice gas approaches, whose underlying architecture also typically runs on classical devices. The Critical Role of the Environment The importance of environmental stimuli must be emphasised in the virtual material approach. Without any external stimuli the virtual material simply reproduces dynamical reaction-diffusion patterning. It is the stimuli provided by external attractants or hazards which force the material to adapt its spatial behaviour. The environmental stimuli are used to specify problem configuration and the final pattern of the material in relation to the stimuli represents the problem solution. The specific mechanism utilised is the diffusion of attractants (or repellents) within the environment. The presence of these stimuli at the periphery of the material provides the impetus for its morphological adaptation. The interaction between environment and the material is two-way, however. When the material migrates towards and engulfs a nutrient source, the diffusion of nutrients from that source is suppressed. This changes the local configuration of chemo-attractant gradients (as demonstrated by the changing concentration gradient profiles in Figure 7e-h as a spanning tree is constructed using the virtual material) which ultimately changes the spatial pattern of stimuli offered to the material. This mechanism is an efficient use of the environment as a spatial storage medium of stigmergic cues and "offloads" (outsources?) some complex computation to the environment. This may explain the reason why slime mould, and its virtual material representation, can perform such complex behaviours without requiring complex nervous system or indeed any neural tissue. Applications, Technologies and Challenges The most obvious application of the virtual material approach is in soft robotics, specifically in the distributed and self-organised generation and control of movement. The attraction of externally controllable, deformable robotic devices that could survive damage and even excision of parts is enticing. The great challenge is in how to move from virtual studies of such materials into development of real prototypes. Chemical interactions and physical forces approximated in computer models are often considered in isolation or in much simpler circumstances than those encountered in the real world and the difficulty of developing real morphological computation substrates cannot be overstated. Some progress has been made in this endeavour, for example the utilisation of self-oscillating chemical reactions to provide motor stimuli in small caterpillar-like polymer gels [32]. Posited materials must have the requisite physical properties, for example resilience to deformation, elasticity, or inherent minimisation properties [30]. However the materials must also allow some means of information propagation within the material, or utilise 75 A Virtual Material Approach to Morphological Computation (a) (b) (c) (d) (e) (f) (g) (h) Figure 7: Construction of a spanning tree by model plasmodium. (a) Small population (particle positions shown) inoculated on lowest node (bottom) growing towards first node and engulfing it, reducing chemo-attractant projection, (b-d) Model population grows to nearest sources of chemo-attractant completing construction of the spanning tree, (e-h) Visualisation of the changing chemo-attractant gradient as the population engulfs and suppresses nutrient diffusion. 76 A Virtual Material Approach to Morphological Computation the physical properties of the material itself for information propagation (for example the oscillatory phenomena within slime mould). Another challenge to material forms of morphological computation is in the "programming", control, output and storage of solutions. Even if certain materials have innate computational properties, how can these abilities be harnessed? The techniques used in prototype unconventional computation schemes may be of help. As noted earlier the problems to be solved by such schemes are typically spatially represented and the output solution (also a spatial pattern) must be converted into a means suitable for storage in classical computers. Hybrid approaches may be used, involving classical computers and algorithms to project the initial spatial patterns (for example using video projection [40]) and interpret the results performed by the non-classical computing substrate. This typically involves transforming, processing, or analysing the final spatial pattern, for example using image analysis techniques to record the positions of nodes and edges in graph optimisation problems [22]. These data may then be stored in a format that is space efficient and suitable for classical processing methods. The concept of control in material based morphological computation is somewhat different from that typically used in classical computing. In classical computing and robotics, which are traditionally subject to top-down development methods, control of a system is maintained by carefully designed feedback processes (typically negative feedback), modulating specific parameter values so that components of the system remain within certain ranges. Unconventional computing methods, however, utilise emergent behaviour which typically arise in systems exhibiting self-organisation and autocatalytic (positive) feedback mechanisms. Control in such systems is a more difficult to define and indeed implement. For example, if the high-level behaviour in a system is an emergent property arising from the interactions between simple components then how do we know which particular components do we feed back information to, and when? Stepney suggested that instead of attempting direct control of a (generic) material computation system we should instead attempt to influence the natural physical behaviour of a system using a system of externally applied fields [48]. As noted earlier, experiments with slime mould suggest it can be influenced using a range of external stimuli, both attractant and repellent and this was also reproduced both globally and locally with the virtual material model [22, 21]. The use of indirect influence of material behaviour, as opposed to imposed top-down control does require a change in philosophy about how we control computation. This would be expected to affect both the design and operation of morphological computation systems. Summary and Conclusions In this article we have adopted a literal interpretation of morphological computation, framing it as the guided control of natural pattern formation mechanisms. Taking the complex computational behaviour of true slime mould as an inspiration we described how the salient features of the organism can be extracted to develop a virtual material approach which uses simple components with local interactions to generate complex 77 A Virtual Material Approach to Morphological Computation emergent patterning behaviour. The virtual material approach uses methods demonstrated in unconventional computing schemes: spatial representation (of both problem configuration and solution), utilisation of natural "material" dynamics and environmental interactions for problem solving, and is amenable indirect influence - rather than control - of material behaviour. The result is a system which exhibits complex behaviour from simple, redundant, component parts and which can be applied to a range of graph minimisation, combinatorial optimisation and distributed robotics applications. The technical challenges faced when moving from such virtual examples to real world materials are significant. But perhaps more significant is the shift in philosophy required during the planning, development and operation of morphological computation systems, moving from notions of precise top-down control towards bottom-up and indirect methods of influencing material behaviour. Acknowledgements This work was supported by the EU research project "Physarum Chip: Growing Computers from Slime Mould" (FP7 ICT Ref 316366) and a UWE SPUR award for early career researchers. Bibliography [1] A. Adamatzky. Developing proximity graphs by Physarum polycephalum: does the plasmodium follow the toussaint hierarchy. Parallel Processing Letters, 19:105–127, 2008. [2] A. Adamatzky. If BZ medium did spanning trees these would be the same trees as Physarum built. Physics Letters A, 373(10):952–956, 2009. [3] A. Adamatzky. Manipulating substances with Physarum polycephalum. Materials Science & Engineering C, 38(8):1211–1220, 2010. [4] A. Adamatzky. Routing Physarum with repellents. The European Physical Journal E: Soft Matter and Biological Physics, 31(4):403–410, 2010. [5] A. Adamatzky and J. Jones. Towards Physarum robots: computing and manipulating on water surface. Journal of Bionic Engineering, 5(4):348–357, 2008. [6] A. Adamatzky. and J. Jones. On electrical correlates of Physarum polycephalum spatial activity: Can we see Physarum machine in the dark? Biophysical Reviews and Letters, 6:29–57, 2011. [7] A. Adamatzky, B. de Lacy Costello, and T. Shirakawa. Universal computation with limited resources: Belousov-zhabotinsky and Physarum computers. International Journal of Bifurcation and Chaos, 18(8):2373–2389, 2008. 78 A Virtual Material Approach to Morphological Computation [8] F. Boschetti and R. Gray. A Turing test for emergence. Advances in applied selforganizing systems, pages 349–364, 2008. [9] A. Boudaoud and S. Chaïeb. Mechanical phase diagram of shrinking cylindrical gels. Physical Review E, 68(2):021801, 2003. [10] J. Buhl, D. Sumpter, I. Couzin, J. Hale, E. Despland, E. Miller, and S. Simpson. From disorder to order in marching locusts. Science, 312(5778):1402, 2006. [11] M. Carlile. Nutrition and chemotaxis in the myxomycete physarum polycephalum: the effect of carbohydrates on the plasmodium. Journal of general microbiology, 63 (2):221–226, 1970. [12] B. de Lacy Costello, N. Ratcliffe, A. Adamatzky, A. Zanin, A. Liehr, and H. Purwins. The formation of voronoi diagrams in chemical and physical systems: experimental findings and theoretical models. International journal of bifurcation and chaos in applied sciences and engineering, 14(7):2187–2210, 2004. [13] M. Dorigo and T. Stutzle. Ant colony optimization, 2004. [14] A. Durham and E. Ridgway. Control of chemotaxis in Physarum polycephalum. The Journal of Cell Biology, 69:218–223, 1976. [15] A. S. G. Paun, G. Rozenberg. DNA computing: New computing paradigms. Texts in Theoretical Computer Science. Springer Verlag, 1998. [16] D. Helbing. Traffic and related self-driven many-particle systems. Reviews of modern physics, 73(4):1067, 2001. [17] D. Helbing, P. Molnar, I. Farkas, and K. Bolay. Self-organizing pedestrian movement. Environment and planning B, 28(3):361–384, 2001. [18] A. Ishiguro, M. Shimizu, and T. Kawakatsu. Don't try to control everything!: An emergent morphology control of a modular robot. In Proceedings of 2004 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 981–985, Sendai, Japan, September 28 October 2 2004. [19] J. Jones. The emergence and dynamical evolution of complex transport networks from simple low-level behaviours. International Journal of Unconventional Computing, 6(2):125–144, 2010. [20] J. Jones. Characteristics of pattern formation and evolution in approximations of Physarum transport networks. Artificial Life, 16(2):127–153, 2010. [21] J. Jones. Towards programmable smart materials: Dynamical reconfiguration of emergent transport networks. Int. Journal of Unconventional Comput., 7(6):423– 447, 2011. 79 A Virtual Material Approach to Morphological Computation [22] J. Jones. Influences on the formation and evolution of Physarum polycephalum inspired emergent transport networks. Natural Computing, pages 1–25, 2011. [23] J. Jones. Multi-agent modelling of the behaviour of Physarum polycephalum. PhD thesis, University of the West of England, UK, 2013. [24] J. Jones and A. Adamatzky. Towards Physarum binary adders. Biosystems, 101(1): 51–58, 2010. [25] J. Jones and A. Adamatzky. Emergence of self-organized amoeboid movement in a multi-agent approximation of Physarum polycephalum. Bioinspiration and Biomimetics, 7(1):016009, 2012. [26] J. Jones and A. Adamatzky. Computation of the travelling salesman problem by a shrinking blob. Natural Computing, 13(1):1–16, 2014. [27] U. Kishimoto. Rhythmicity in the protoplasmic streaming of a slime mould, Physarum polycephalum. The Journal of general physiology, 41(6):1223–1244, 1958. [28] J. Koza and R. Poli. Genetic programming. Search Methodologies, pages 127–164, 2005. [29] I. Lagzi, S. Soh, P. Wesson, K. Browne, and B. Grzybowski. Maze solving by chemotactic droplets. Journal of the American Chemical Society, 132(4):1198–1199, 2010. [30] T. Lobovkina, P. Dommersnes, S. Tiourine, J. Joanny, and O. Orwar. Shape optimization in lipid nanotube networks. The European Physical Journal E: Soft Matter and Biological Physics, 26(3):295–300, 2008. [31] T. Lobovkina, I. Gozen, Y. Erkan, J. Olofsson, S. Weber, and O. Orwar. Protrusive growth and periodic contractile motion in surface-adhered vesicles induced by ca2+ gradients. Soft Matter, 6(2):268–272, 2010. [32] S. Maeda, Y. Hara, T. Sakai, R. Yoshida, and S. Hashimoto. Self-Walking Gel. Advanced Materials, 19(21):3480–3484, 2007. [33] M. Matsushita, J. Wakita, H. Itoh, K. Watanabe, T. Arai, T. Matsuyama, H. Sakaguchi, and M. Mimura. Formation of colony patterns by a bacterial cell population. Physica A: Statistical Mechanics and its Applications, 274(1):190–199, 1999. [34] M. Mitchell. An introduction to genetic algorithms. MIT Press, Cambridge, MA, USA, 1996. ISBN 0-262-13316-4. [35] T. Nakagaki, S. Uemura, Y. Kakiuchi, and T. Ueda. Action spectrum for sporulation and photoavoidance in the plasmodium of Physarum polycephalum, as modified differentially by temperature and starvation. Photochem. Photobiol., 64(5):859–862, 1996. 80 A Virtual Material Approach to Morphological Computation [36] T. Nakagaki, H. Yamada, and A. Toth. Intelligence: Maze-solving by an amoeboid organism. Nature, 407:470, 2000. [37] T. Nakagaki, H. Yamada, and T. Ueda. Interaction between cell shape and contraction pattern in the Physarum plasmodium. Biophysical Chemistry, 84(3):195–204, 2000. [38] R. Pfeifer and F. Iida. Morphological computation: Connecting body, brain and environment. Japanese Scientific Monthly, 58(2):48–54, 2005. [39] V. Privman, M. Arugula, J. Halámek, M. Pita, and E. Katz. Network analysis of biochemical logic for noise reduction and stability: A system of three coupled enzymatic and gates. The Journal of Physical Chemistry B, 113(15):5301–5310, 2009. [40] N. G. Rambidi. Biologically inspired information processing technologies: Reactiondiffusion paradigm. Int. J. Unconventional Comput., 1(2):101–121, 2005. [41] D. Reyes, M. Ghanem, G. Whitesides, and A. Manz. Glow discharge in microfluidic chips for visible analog computing. Lab Chip, 2(2):113–116, 2002. [42] C. Reynolds. Flocks, herds and schools: A distributed behavioral model. In ACM SIGGRAPH Computer Graphics, volume 21, pages 25–34. ACM, 1987. [43] E. Ronald, M. Sipper, and M. Capcarrère. Design, observation, surprise! a test of emergence. Artificial Life, 5(3):225–239, 1999. [44] T. Saigusa, A. Tero, T. Nakagaki, and Y. Kuramoto. Amoebae anticipate periodic events. Phys. Rev. Lett., 100(1), 2008. doi: 10.1103/PhysRevLett.100.018101. [45] T. Shirakawa and Y.-P. Gunji. Computation of Voronoi diagram and collision-free path using the Plasmodium of Physarum polycephalum. Int. J. Unconventional Computing, 6(2):79–88, 2010. [46] T. Shirakawa, A. Adamatzky, Y.-P. Gunji, and Y. Miyake. On simultaneous construction of voronoi diagram and delaunay triangulation by Physarum polycephalum. International Journal of Bifurcation and Chaos, 19(9):3109–3117, 2009. [47] S. Stephenson, H. Stempen, and I. Hall. Myxomycetes: a handbook of slime molds. Timber Press Portland, Oregon, 1994. [48] S. Stepney. The neglected pillar of material computation. Physica D: Nonlinear Phenomena, 237(9):1157–1164, 2008. [49] S. Takagi and T. Ueda. Emergence and transitions of dynamic patterns of thickness oscillation of the plasmodium of the true slime mold Physarum polycephalum. Physica D, 237:420–427, 2008. 81 A Virtual Material Approach to Morphological Computation [50] A. Takamatsu, T. Fujii, and I. Endo. Control of interaction strength in a network of the true slime mold by a microfabricated structure. BioSystems, 55:33–38, 2000. [51] S. Tsuda and J. Jones. The emergence of complex oscillatory behaviour in physarum polycephalum and its particle approximation. In Artificial Life XII, pages 698–705, 2010. [52] S. Tsuda and J. Jones. The emergence of synchronization behavior in Physarum polycephalum and its particle approximation. Biosystems, 103:331–341, 2010. [53] S. Tsuda, M. Aono, and Y.-P. Gunji. Robust and emergent Physarum logicalcomputing. BioSystems, 73:45–55, 2004. [54] S. Tsuda, K. Zauner, and Y. Gunji. Robot control with biological cells. BioSystems, 87(2-3):215–223, 2007. [55] S. Tsuda, J. Jones, and A. Adamatzky. Towards Physarum engines. Applied Bionics and Biomechanics, in press, 2012. [56] T. Ueda, K. Terayama, K. Kurihara, and Y. Kobatake. Threshold phenomena in chemoreception and taxis in slime mold Physarum polycephalum. The Journal of general physiology, 65(2):223–34, February 1975. ISSN 0022-1295. [57] T. Umedachi, T. Kitamura, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. A modular robot driven by protoplasmic streaming. Distributed Autonomous Robotic Systems, 8:193–202, 2009. [58] A. Zanin, A. Liehr, A. Moskalenko, and H. Purwins. Voronoi diagrams in barrier gas discharge. Applied Physics Letters, 81:3338, 2002. [59] K. Zauner and M. Conrad. Parallel computing with dna: toward the anti-universal machine. Parallel Problem Solving from Nature PPSN IV, pages 696–705, 1996. [60] S. Zhang. Fabrication of novel biomaterials through molecular self-assembly. Nature Biotechnology, 21(10):1171–1178, 2003. 82 Abstract: In the existing contributions to morphological computation, it is found that the morphological property should play a role in interfacing the internal control mechanism of a system with the outside environment. This property not only gives the connection between them, but also mediates, reconciles, and integrates them. However, the problem of what distinguishes the morphological property of the system from that of usual complex control systems seems to remain. As a possible solution, we consider the notion of heterarchy, which is the interplay between hierarchies such as agent and environment. Our discussion reveals that the properties of heterarchical and morphological systems are closely related to each other. For the development of morphological computation, some investigations of the heterarchical property are introduced, and some proposals are submitted. A Review of Morphological Computation from a Perspective of Heterarchy A Personal Definition of Morphological Computation The relationships between body and mind, the physical and the logical, hardware and software, etc. encompass myriads of issues in various research areas. Morphological computation may be able to improve upon software-side problems by using hardwareside solutions. The term morphology in morphological computation means a solution of the boundary condition definition problem such as the frame problem, that is, the most difficult problem in the field of artificial autonomy. Here, I introduce my personal definition of morphological computation: Morphological computation is a solution to reduce the complexity of a computational representation for an environment by introducing a complex interface between the system and the environment. The robot dog mini-dog [17], which is an example of a morphological computation system, consists of legs made of an elastic spring mechanism and a rigid rod-shaped body connecting the legs. Although the robot does not have complex control functions, it can walk in the same manner as real dogs. In this mechanism, the movement of the center of gravity reconciles the independent dynamics of the legs. As a result, a limit cycle appears as the rhythmical motion of the legs. Since the dynamical system of the dog is a multiple degree-of-freedom system, it requires the huge amount of conditions to control individual element. Here, the synthetic behavior of the dog is generated by itself without the control difficulty. Such an appearance of synthesis is called emergence in the research field of complex systems. The notion of emergence is usually discussed with respect to multiple degree-of-freedom systems such as cellular automaton [26] or diffusion reaction systems [10]. It is conventionally considered to be a result from the local interaction between elements following defined rules. However, there is no powerful theory that combines emergent behavior with local rules. Therefore, it is difficult to control the whole system by given local rules. In the mini-dog case, however, by actively utilizing uncontrollability caused by complex representations of the environment such as the friction of the floor and the dynamics of the center of gravity, the natural movement of walking can be achieved by a simple control function. From this point of view, it can be said that morphological computation can provide a framework for controllability of a system in a complex environment. Because this explanation is thought to be insufficient for the purpose of this book, a deeper discussion is needed. In the above-mentioned examples of morphological computation, the systems can regulate the dynamics at the whole-system level, e.g. the movement of the center of gravity. This becomes possible when the behavior of a system converges to an attractor. Since a human has states such as running, walking, stepping, and so on, multiple attractors may exist in real systems. How can we model and treat such systems with multiple attractors? This transition behavior between attractors was discussed by Tsuda [25, 1]. Here, to explain this problem of transition among attractors, I use the example of a treadmill, a running exercise machine that I often use at the sports gym to improve my flabby body. From my experience, a speed of less than 6 km/h is comfortable for walking, though it depends on the degree of tilt (set by controlling the resistance). However, if the speed exceeds 7km/h, we have to run. Here, there is a middle zone of 6–7 km/h where 84 A Review of Morphological Computation from a Perspective of Heterarchy we can both run and walk. We usually find a comfortable point to change state from walking to running in this zone. Unfortunately, even if the treadmill's speed is increased continuously, we would never perceive the suitable speed at which the natural transition from running to walking takes place. The transition point is not autonomically evident; we just have to define it without an exact basis. In other words, the bifurcation point is heuristically defined by ourselves. The transition from the walking state to the running state corresponds to the transition between the attractors. If the ground moves at the same speed as our legs, the running state is stabilized. Hauser et al. [6] argued that this type of transition among attractors is manipulated by brain level functions. Here, we can see the two hierarchies in morphological computation: 1. the hierarchical relationship between the system and its environment and 2. the relationship between subsystems and the "wholeness", in other words, between the body and the brain. If we suppose that a system is equivalent to a body, then the representation of intelligence is similar to existing artificial intelligence. Thus, if morphological computation explores the role of inconsistency between the body and the system, it simultaneously gives a perspective of systems by the notion of heterarchy. Morphological Computation and Heterarchy Here, I describe the notion of heterarchy and explain my reason for using it to discuss morphological computation. Our research group has been studying the notion of heterarchy as dynamical hierarchy. Heterarchy is a term used to explain the functional structure model on a neural network. McCulloch [15] proposed heterogeneous loops in the neural circuit representing brain functions. The heterogeneous loops means there exists a variety of the feedback and feedforward loops to make cognitions, decisions and actions, and they interact each other. Unfortunately, the novelty of this idea was not easy to understand when the notion of heterarchy has been proposed. To understand this, imagine cognitive experiments. A cognitive experiment is defined as a stimulus and reaction pair. If we suppose only one loop in the brain circuit, we can consider only one pair of them. However, a heterogeneous loop represents a set of stimuli and a set of reactions. Then, the experimental results are not distinguishable from past effects. Usually, with a sufficient training phase, these effects are negligible. In contrast, physiological approaches to brain function revealed larger loops via ganglia networks [27]. This implies the existence of heterogeneity in neural signaling loops. By the remarkable development in recent methods of recoding patterns of brain connectivity and their quantitative analysis tools [21], more complex neural activity can be investigated and it would became possible to find the actual heterarchical structure. The term heterarchy became conventional when sociologists studying economic organization started to use the term. Stark [22, 23] introduced the notion of heterarchy to define the robustness of an organization such as a company or a government. If a company has a single value estimation mechanism that defines what is good, the company will collapse by a huge perturbation from the external environment. He thought that multiple, contrary senses of order coexist in a robust company and so that is the heter85 A Review of Morphological Computation from a Perspective of Heterarchy A B C A B C X X X Z Figure 1: Schematic diagrams from simple hierarchy to heterarchy. archical organization. However, it is not easy to maintain such dissonance in the usual organization because it is difficult to create enough innovation to establish a new sense of order. Stark concluded in his theoretical consideration that positive encouragement to formation of new ideas, including contradiction, is a possible solution to these problems. To make the concept of heterarchy understandable, consider the schematic representation of heterarchy in Figure 1. A usual hierarchy is shown in Figure 1a. An element X is indicated by intension as a wholeness. For example, the indication corresponds to the defined value estimation function in an organization or a brain function representation caused by a network loop. Usually, a hierarchy is described as a set that keeps consistency between intent and extent. The consistency indicates the equivalence between intentional expression (e.g. 2x) and extensional expression (e.g. {2, 4, 6, 8, * * * }). Conventionally, this consistency is thought to be necessary if we imagine the superposed 86 A Review of Morphological Computation from a Perspective of Heterarchy hierarchies, e.g. particles, cells, organisms, animals, society and so on. Figure 1b shows a case of multiple wholeness. To understand this situation, consider the temporal transition of management policy such as A ! B ! C. According to the transition of policy, the valuations are A(X) ! B(X) ! C(X). Note that, since there is only a single set considered, the simultaneous existence of multiple wholenesses is prohibited in this case. Spencer-Brown [20] introduced this transition to avoid the deadlock induced from the static relationship between intension and extension in a logic form. The situation is thought as a government that is temporally replaced by another one, despite the people in the country do not change. Figure 1c shows the case of multiple sets and wholeness. Here, the assumption is made that the sets are growing such that {X} ! {X,Z}. To permit this situation, we have two kinds of wholeness transitions: A ! B and A ! C. The former case is equal to the case shown in Figure 1b, but latter case is different from a simple time transition. This discussion is closely related to the concept of autopoiesis [14, 11], which is a system autonomously generates its boundary condition and definition of its behavioral rules. However, the concept of autopoiesis has no defined interplay between intensional and extensional changes of a system. In other words, boundary reconfiguration and time transition are distinguished as different problems. This definition cannot solve the problem of infinite regression because individual transitions require a basis such that the time transition requires a fixed state space and a definition of the boundary condition requires a static snapshot of time. Our motivation to study heterarchy is to explore the interplay of these two kinds of transitions and what will evolve from this dynamical framework as shown in Figure 1d. Gunji described an example that is a heterarchy of human being [5]: Consider about a man which is not only a member of his family, but also a member of a company. If he goes to work on his day off, his action would benefit his company, but it would be detrimental to his family. Then, his action simultaneously affects both of two levels, his family and company. However, when the man is in the hallway hesitating to go work, his wife decide to hide her disappointment and wishes him farewell with a smile. Her action changes the evaluation of the man's action in the family. This interplay between the man and his family benefits the company as acceleration of his motivation. After his work, the company would give him a vacation. Therefore the interplay between hierarchies makes some innovative change and keeps the robustness of the company and the family. As mentioned before, the problem I discuss here is the morphology that must be fixed in the existing morphological computation examples, where the term morphology means physical parameters of the springs (e.g. stiffness, damping, etc.) and the characterization of the materials used for instance. Thus, their contributions are available only for a fixed environment where the task and boundary conditions are fixed. Reconsidering the definition of morphological computation, one concludes that the kind of morphology should have an unfixed, dynamic characterization because morphology itself is not a function but a function basis. In other words, morphological computation should be understood as a dynamical system based on the above-mentioned heterarchical interplay between two layers. As heterarchical dynamics, we combined the notion of heterarchy with robustness [8], which is the structural adaptability to outside turbulence [5]. Since the heterarchical 87 A Review of Morphological Computation from a Perspective of Heterarchy structure is based on the diversity of subsystems and the dissonance among them we think that a heterarchical system can realize robustness. How do such systems based on morphological computation maintain dissonance, diversity? In my previous research I concluded that internal fluctuation of a natural system is caused by a regulation process for inconsistency between whole-system and subsystem level dynamics [18]. Thus, we have to investigate whether the flexibility of morphology can or cannot maintain the diversity of dynamics, which corresponds to the diversity of attractors, to propose a way to make existing artificial systems robust by using morphological computation. Positively Used Indefiniteness Induced from Morphological Properties Firstly, I would like to start from the concept of internal measurement, which is helpful to understand what we will discuss in this section. Internal measurement [13] is the simple hypothesis that an internal observer simultaneously has properties of both an observer and a performer. Usually, when a phenomenon is described, an external observer is implicitly assumed because an exact description of a phenomenon requires indication of both the object and the environment. However, we cannot know the environment completely in the real world because we are not only an observer but also a performer in the world. We think it is quite important to consider internal measurement to understanding the nature of life. Unfortunately, it is not easy to construct a formal theory of internal measurement because, if we simply assume internal measurement, we falls into an infinite regress of verification for the observation. For instance, consider the situation when we recognize a red apple. To distinguish the apparent object is an apple, we always match the stimulus from visual cortex to the memory. In contrast, if there is no memory about apple, we cannot recognize apple. Usually, we do not concern about this cognitive mechanism because only the apple as the result of the cognition rises up to the conscious, and that is a simple measurement. In the case of internal measurement, the cognitive system including the memories has to be simultaneously defined because the realized world including a observer. Since the cognition of the cognitive system is a self-referential action, it leads the infinite regress as the infinite loop of cognition, i.e. the cognition of the cognition of the cognition... Nonetheless, we are living without infinite regress. Therefore, It can be said that we always make a tentative boundary to avoid the infinite regress. Since the tentative boundary is not a strict boundary, it can be reconciled with the recognized object. The actual internal measurement is realized based on the interplay between the tentative cognitive system and the recognized object. This discussion is available if there exists an interplay between hierarchies such as internal behavior and the external world, that is, if we consider heterarchy. In the example the business man in the previous section, the indefiniteness of tentative boundary, i.e. cooperation of his family, acts positively because it benefits both of the family and the company. This positively used indefiniteness is the key idea in this section. Some types of representation for the heterarchical property have been proposed. Gunji and Kamiura defined a categorical expression of observational heterarchy that is a mixture of internal 88 A Review of Morphological Computation from a Perspective of Heterarchy and external representations [3, 9]. Gunji and I studied dynamical systems based on positively used indefiniteness derived from heterarchy [5, 4, 18]. A theoretical analysis of the interplay between brain dynamics and body dynamics on human awareness has also recently been established [19]. From the above approaches, I would like to focus on a dynamical system based on the dynamic relationship between time and space [5, 18]. Figure 2 shows the summary of the time-space re-entrant model and its behavior (The original figure is presented in [18]). The model describes a mixture of temporal dynamics and spatial dynamics, which represents wholeness growing as mentioned in the previous section. Re-entrant means sequentially applied time and space dynamics with entrant operations F and G. The schematic diagram of the re-entrant system is displayed in Figure 2a. As shown in Figure 2b, we constructed a time-space re-entrant dynamical system with a logically twisted computation direction, where the spatial direction means from the significant digit to the last digit and where a bar indicates a real-valued binary expression. Grim et al. [2] demonstrated that the dynamical property of a self-referential sentence is represented as a chaotic dynamical system, which is called a chaotic liar. By applying this re-entrant form to a chaotic liar, we obtained a pulse-like behavior, which is different from chaotic behavior (see Figure 2c). The most remarkable characteristic of the system is the structure of the return map, which shows an entrained fractal structure such as Figure 2d. The twisted digit and time direction induce infinitely folded layers in the form of a Cantor set. Statistically, the system exhibits a Lévy flight power-law behavior, which is found in a lot of natural dynamical systems. Once we focus on the time transition or spatial boundary reconfiguration, we stack into an infinite regression represented by chaotic indefiniteness. However, if we consider baseless indefiniteness corresponding to a mixture of time and space, robust reasoning for decision making becomes possible in the form of resulting pulse-like behavior based on fractal entrained dynamical structure. By reconsidering morphological computation on the scheme of a time-space re-entrant system, the morphological property of a system should play the role of twisting space to time. Remember the case of the mini-dog. From the viewpoint of conventional robotic construction, we have to consider individual elastic joint control, frictional effects, and sophisticated synthetic action planning of several operations, but the system finally falls into a frame problem. By considering the morphological properties of the legs, the control mechanism is not precise but the overall synthetic control of the body can achieve smooth behavior. This mechanism is very close to that of the time-space re-entrant system whose boundary indefiniteness is twisted into temporal dynamics. Thus morphological computation and heterarchy are closely related and thus should be considered together. Possible Applications of Morphological Computation Let us consider what applications of morphological computation are possible in the future. As mentioned before, the morphological property could be a basis in which heterarchical interplay occurs. Essentially, the investigation of the possible developments of animal-like robots led to the idea of morphological computation. However, the innovative 89 A Review of Morphological Computation from a Perspective of Heterarchy W G F   

G F   Figure 2: Time-space re-entrant model. Figures have been taken from [5] and [18]. 90 A Review of Morphological Computation from a Perspective of Heterarchy aspect of morphological computation is not related to the technical solutions of building a robot control mechanism, because unconventional computation has the same goal. Thus, morphological computation should help us discover a robust adaptation mechanism for animals. In addition, the results of morphological computation would aid in exploring the meaning of the interaction between two levels (e.g., the brain and the body, organization and function, and system and environment). The two fields of artificial intelligence and artificial life have been competing over the priority between the brain and the body. Several challenging results in those fields imply the necessity of combining them and establishing a new research field of "complex intelligence" and/or "intelligent complexity." A trinity of the brain, the body, and the environment in morphological computation will mark the first step in such a crossover paradigm of computation. The exploration of a new computing paradigm is a matter of some urgency. Recently, there have been many studies of systems deployed over computer networks such as the Internet, whose structures are rather amorphous. Because these systems continue to grow rapidly, the number of related problems is also increasing. Some improvements to understanding such complex networks have been introduced from studies related to biology [24, 16]. Alternatively, there are applications in which intelligence is introduced into components of a system [12, 7]. These approaches evoke similarities to the competition between artificial intelligence and artificial life. Thus, an investigation of the role of morphology may also improve upon methods to solve such problems. In actual implementation of probabilistic algorithms, we have to use some kind of random generator or chaotic oscillator to realistically model a switching mechanism for instant selection. However, our previous results imply that pulse-like behavior is more suitable for representing a natural switching mechanism than chaotic (random) noise. Bibliography [1] R. M. Füchslin, A. Dzyakanchuk, D. Flumini, H. Hauser, K. J. Hunt, R. H. Luchsinger, B. Reller, S. Scheidegger, and R. Walker. Morphological computation and morphological control: steps toward a formal theory and applications. Artificial life, 19(1):9–34, 2013. [2] P. Grim, G. Mar, and P. Denis. The Philosophical Computer. The MIT press, Cambridge, MA, 1994. [3] Y.-P. Gunji and M. Kamiura. Observational heterarchy enhancing active coupling. Physica D: Nonlinear Phenomena, 198(1-2):74 – 105, 2004. ISSN 0167-2789. [4] Y.-P. Gunji, K. Sasai, and M. Aono. Return map structure and entrainment in a time-state-scale re-entrant system. Physica D., 234:124–130, 2007. [5] Y. P. Gunji, K. Sasai, and S. Wakisaka. Abstract heterarchy: Time/state-scale re-entrant form. Biosystems, 91(1):13–33, 2008. 91 A Review of Morphological Computation from a Perspective of Heterarchy [6] H. Hauser, A. J. Ijspeert, R. M. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological Cybanetics, 105(5–6):355–370, 2012. [7] J. Hendler. Agents and the semantic web. Intelligent Systems, IEEE, 16(2):30–37, 2001. [8] E. Jen. Stable or robust? What's the difference? Complexity, 8(3):12–18, 2003. [9] M. Kamiura and Y.-P. Gunji. Robust and ubiquitous on-off intermittency in active coupling. Physica D: Nonlinear Phenomena, 218(2):122 – 130, 2006. ISSN 01672789. [10] K. Kaneko. Globally coupled chaos violates law of large numbers. Phys. Rev. Lett., 65:1391–1394, 1990. [11] N. Luhmann. Social Systems. Stanford University Press, 1995. [12] P. Maes. Agents that reduce work and information overload. Communications of the ACM, 37:30–40, 1994. [13] K. Matsuno. Protobiology: Physical Basis for Biology. CRC Press, Boca Raton, 1989. [14] H. Maturana and F. Varela. Autopoiesis and Cognition: The Realization of the Living. D. Reidel, Dordrecht, 1991. [15] W. McCulloch. A heterarchy of values determined by the topology of nervous nets. Bulletin of Mathematical Biology, 7(2):89–93, 1945. [16] M. Meisel, V. Pappas, and L. Zhang. A taxonomy of biologically inspired research in computer networking. Computer Networks, 54:901–916, 2010. [17] R. Pfeifer, F. Iida, and G. Gómez. Morphological computation for adaptive behavior and cognition. International Congress Series, 1291(22):22–29, 2006. [18] K. Sasai and Y. P. Gunji. Heterarchy in biological systems: A logic-based dynamical model of abstract biological network derived from time-state-scale re-entrant form. Biosystems, 92(2):182–188, 2008. [19] K. Sonoda, K. Kodama, and Y.-P. Gunji. Awareness as observational heterarchy. Frontiers in Psychology, 4(686):1–12, 2013. [20] G. Spencer-Brown. Laws of form. George Allen & Unwin, London, 1969. [21] O. Sporns. Contributions and challenges for network models in cognitive neuroscience. Nature Neuroscience, 2014. [22] D. Stark. Heterarchy: Distributing Authority and Organizing Diversity. Oxford University Press, Oxford, 1995. 92 A Review of Morphological Computation from a Perspective of Heterarchy [23] D. Stark. Heterarchy: Distributing authority and organizing diversity. In J. I. Clippinger, editor, The Biology of Business: Decoding the Natural Laws of Enterprise, pages 153–179. Jossey-Bass Publishers, San Francisco, 1999. [24] R. Sterritt. Autonomic computing. Innovations Syst Softw Eng, 1:79–88, 2005. [25] I. Tsuda. Hypotheses on the functional roles of chaotic transitory dynamics. Chaos: An Interdisciplinary Journal of Nonlinear Science, 19(1):015113, 2009. [26] S. Wolfram. Cellurar automata. Los Alamos Science, 9:2–21, 1983. [27] H. H. Yin, S. B. Ostlund, and B. W. Balleine. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. European Journal of Neuroscience, 28(8):1437–1448, 2008. 93 Abstract: Heuristic Bio-Robotics is a new paradigm for understanding and reproducing animals' adaptive behavior. In this approach, Morphological Computation is a very important design principle that provides quick response, reduction of formal computation, and adequate coordination transformations for multiple tasks and open environment. Morphological Computation and Heuristic Bio-Robotics Heuristic Bio-Robotics Approach Amazing progress in robotics in these decades has been strongly supported by the formal computation. A robot and its environment are formally modeled, and based on these models, a control scheme is derived for realizing desired behavior. Since everything is formalized, we can prove the overall stability of the system and estimate performance before we actually conduct real-world experiments. Robot experiments are conducted for validation of the scheme and/or for execution of the task. The behavior of the system largely relies on the formalization, and so far, adaptability of existing robots is far inferior compared to the animals. Animals can behave adaptively in open/unknown/clutter environment. They can adapt to large changes of the task and environment. They can react to changes rapidly, and can easily learn new situations based on old knowledge. All of these properties are not processed in the brain formally, but it seems that other elements such as compliance of the limbs and local feedback control play a great role. However, these hypotheses cannot be validated by formalization because the task and environment of animals are basically open and cannot be formalized. To understand and reproduce animal-like adaptability, a formal approach is not so effective. Instead, we can adopt an biological and experimental approach: by observing animals' bodies and the behavior carefully, we can make hypotheses on the body structure and peripheral control contributing to adaptability; then, we can implement these elements to a real robot and test them in the real environment. This is, what we call, the Heuristic Bio-Robotics approach in which robots are not merely tools to validate (off-line) computation, but tools to explore new scientific findings. Morphological computation In the Heuristic Bio-Robotics approach, we carefully observe animals' body structure and make hypotheses for realizing adaptive behavior. The design principle for adaptive behavior is Morphological Computation. Structural compliance such as preflex [1, 3], series elastics, and minor loops realized by peripheral feedback, just to name a number of examples, which can react immediately to the stimuli from the environment. These components are not formally represented by the central (nerve) system. Morphological computation represents local physical entities, peripheral feedback, any computationequivalent components related to adaptability hidden from the central system. Morphological computation is one of the principles for adaptability in the context of Heuristic Bio-Robotics. The peripheral computation is expected to be effective not only for reducing computation of the central system, but for adaptability to wider a variety of tasks. The environments of the animals are open, and they should deal with several tasks. Therefore, the peripheral computation should be a preprocess to generate information adaptively over wide variety of tasks and environments. It should not be optimized for a single task, but should be moderate for many tasks. Otherwise, the animal is sensitive to changes of tasks 95 Morphological Computation and Heuristic Bio-Robotics and environments and is not evolutionally selected. Yet, we do not have any principle to design such peripheral control. Instead, we have to observe biological systems very carefully, find many hypotheses on such peripheral computation, and test them with real robots. This is the core idea of Heuristic Bio-Robotics. In the following three sections, we will discuss on three important insights of Morphological Computation, (1) quick response by peripheral computation, (2) reduction of computation of the central system, and (3) coordination transfer for wide variety of tasks and environments. Quick Response by Peripheral Computation Firsly, we discuss on quick response realized by pheriperal control. Morphological computation is computation-equivalent physical processes and/or local feedback in the peripheral, and can reduce information flow to the central system. As a result, it can realize quick response to stimuli from the environment. This feature is important, especially, for fast locomotion. For example, compliance of the legs of animals largely contributes to efficiency and stability of locomotion, known as preflex [1, 3]. Cham et al. realized cockroach-like fast locomotion by utilizing such preflex [2]. The compliant structure was designed by exploratory simulation. Since the preflex in [2] is realized by fixed hardware, it cannot be changed to modulate locomotion. If the compliant structure can be changed, movement may be modulated such that animals/robots modulate their behavior by changing balance of muscular skeleton system. The monopod shown in Figure 1 has similar muscular skeleton structure to a human's lower limb, which has 9 muscles [4], which is designed to show how the balance of the system can modulate the overall behavior. By many experimental trials, we could explore the function of muscles, and found that the robot can modulate the jumping direction by changing tension of bi-articular muscles. In short, quick response should be modulated by changing morphology to realize adaptability. Reduction of Computation of the Central System Second important point of Morphological Computation is reduction of computation of the central system. It is strongly related to the first feature since quick response can be generated by peripheral computation, and as a result, not much information has to be transferred to the central system. The simplest example is the role of elastics in series in walking. During walking, a foot never steps on the same position: therefore, the spring constant of the terrain is not unique. If the robot is stiff, it should deal with such fluctuation of the spring constant formally. If it has series elasticity, the overall spring constant of the terrain and the robot does not change so much, the condition that the robot should deal with is relaxed to some extent. 96 Morphological Computation and Heuristic Bio-Robotics Figure 1: A jumping robot with anthropomorphic muscular skeleton structure. Muscles #1 (iliacus) and #2 (gluteus maximus) are monoarticular muscles driving the hip joint. Muscles #3 (vastus lateralis) and #4 (popliteus) drive the knee and muscles #7 (tibialis anterior) and #8 (soleus) drive the ankle joint, respectively. Muscles #5 (rectus fomoris), #6 (hamstring muscles) and #9 (gastrocnemius) are biarticular muscles that drive not only one but also two joints(the figure and the caption are from [4]). A little more complicated example to reduce central computation can be found in the attachment for traditional industrial robots: a RCC (Remote Center Compliance) wrist (Figure 2). The RCC wrist consists of several elastic components so that the compliance center is placed at the tip of a grasped peg. Imagine the grasped peg touches one face of the hole, and it gets reaction force from the face. If the reaction force vector passes through the compliance center, the peg moves only in translation. If the reaction force generates moment along the center, the peg rotates along the compliance center. These reactions are generated automatically from the compliant property of the hand, and achieve the insertion task without recognizing position and orientation of the peg formally by the central system. The muscular-skeleton structure for jumping introduced in the previous section is also a very nice example for reducing central computation: the coordinated movement of the joints is generated automatically by the morphology of the musclar skeleton system. Coordinate Transform Last but not least, we will discuss on the coordinate transfer for wide variety of tasks and environments. If the preprocessing brought by the peripheral computation is optimized for a particular task, it may not be good for other tasks. It should not be overfit to only one task, but should be marginal to a certain variety of tasks and environments. 97 Morphological Computation and Heuristic Bio-Robotics Figure 2: A schematic illustration of RCC hand The peripheral computation provides coordinate transforming to reduce the computation of the central system, but the cooridates should not be specialize to a certain task. The morphological computation should provide moderate coordinate transforming for a variety of tasks and environments so that the central system can utilize. Traditional design strategies are optimized for a single task. If the class of tasks and environments is estimated before designing, we can marge their indexes by taking weighted sum for the optimization. But, it is still very difficult to appropriately determine the weight of each task in the natural environment. The other way to design an adequate coordinate transform is to learn from biology: Heuristic Bio-Robotics approach. Since animals are evolved so that they can survive in open environment, their peripheral computation is adequate for surviving in the real world. We can pick up many hypotheses on the body structure and the peripheral control network from biology and test them by realizing robots in real environment. For example, let's look at the muscular-skeleton system of the human's arm. It consists of many bones, muscles, and ligaments and provides the structural compliance contributing to the adaptability. This structural compliance is an adequate coordinate transform of the arm robot. We reproduce the structure as much as possible, and test its ability in the real world (Figure 3). For example, we showed some experimental results on door-opening [5]. If the robot is placed in the same position/orientation with respect to the door (knob), the door-opening task can be described as a formal problem consisting of trajectories of position and force. Then, we adopt electric motors that are suitable for realizing formal trajectories. However, in the real situations, the robot is not placed precisely with respect to the door. The door-opening task is trivial for a human, but 98 Morphological Computation and Heuristic Bio-Robotics Figure 3: Anthoropomorphic upper arm driven by pneumatic artificial muscles cannot be easily described in a formal way. If we use an anthropomorphic muscularskeleton robot, it can open the door utilizing its structural compliance. Note that the door is essentially designed for human-use [6], therefore, it should be easy for the robot to deal with the door since it has similar body structure as humans. In other words, the door-opening task is trivial if the robot has similar morphology as humans. The robot can utilize the two points of the morphological computation, quick response of peripheral computation and reduction of computation of the central system, and can realizes the door-opening task in which a simple controller is applied in the central system. Interaction Between a Human and a Robot It is important for robots to have the same morphology as a human for communication. If the robot has the same morphology, we can share common senses on our bodies. Let's look at the example of the anthropomorphic robot arm again. Since the robot has the same morphology as a human, it is very simple to teach the robot to open the door. Especially, we utilize our body structure to open the door, we do not have to be explicitly aware of how our morphology works for the task, but simply teach the sequence of sub tasks: grab, turn, push, and follow the door motion [5]. Note that we and the robot share a similar morphological structure: morphological computation. In this sense, to realize natural interaction between humans and robots, we have to be aware how morphological computation works since it is not formally expressed in the central system. 99 Morphological Computation and Heuristic Bio-Robotics Conclusion By extracting possible morphological computation by observing biological system, we can realize adaptive and versatile robots. What is more interesting is, by implementing hypotheses on the biological morphology into robots and test them in the real world, we can find new scientific evidences on biological intelligence (adaptation). This is Heuristic Bio-Robotics. We can use robots not for merely validating the formal computation, but for digging out new scientific findings. The field is expected for the new future development. Bibliography [1] I. E. Brown and G. E. Loeb. A reductionist approach to creating and using neuromusculoskeltal models. In J. M. Winters and P. E. Crago, editors, Biomechanics and Neural Control of Posture and Movement, pages 148–163. Springer–Verlang New York, 2000. [2] J. G. Cham, S. A. Bailey, J. E. Clark, R. J. Full, and M. R. Cutkosky. Fast and robust: Hexapedal robots via shape deposition manufacturing. International Journal of Robotics Research, 21(10–11):869–882, 2002. [3] R. J. Full. Biological inspiration: Lessons from many-legged locomotors. In J. M. Hollerbach and D. E. Koditschek, editors, Robotics Research 9, pages 337–341. Springer London, 2000. [4] K. Hosoda, Y. Sakaguchi, H. Takayama, and T. Takuma. Pneumatic-driven jumping robot with anthropomorphic muscular skeleton structure. Autonomous Robots, 28(3): 2010, 2010. [5] K. Hosoda, S. Sekimoto, Y. Nishigori, S. Takamuku, and S. Ikemoto. Anthropomorphic muscular-skeletal robotic upper limb for understanding embodied intelligence. Advanced Robotics, 26(7):729–744, 2012. [6] D. Norman. The Design of Everyday Things. Basic Books, 2002 (Reprint version). 100 Abstract: This article contains our perceptions and comments on the literature concerning morphological computation. We argue that, if there is no computation without intelligence and if morphology is the embodiment of this intelligence, the usage of the term "morphological computation" is trivial. Therefore, another term should be used to define this new discipline, encompassing all its aspects. We have come up with a new term, "Morphological Intelligence," and have identified three subfields, namely, "Morphological compatibility", "Morphological intelligence transformation," and "Morphological intelligence optimization." We believe that the ramifications of this new terminology will help us better understand the discipline. In this paper, we explain this concept, using our new terminology, in the context of locomotion. Morphology: A Concrete Form of Intelligence Introduction We, as humankind, have been interested in the behavior of physical bodies exposed to forces or displacements since ancient times. We identify and name the physical properties of objects in nature in order to better understand the world around us; today we call the sum of these features, "Morphology." Humans manipulate the morphology of objects to better utilize these objects for a variety of tasks. We study the behaviors of these objects in the discipline of mechanics, which uses mathematics and simplified governing equations to understand complex systems. Although we use many simplifications in mathematical methods, these equations provide us with strong instruments to obtain approximate solutions in most cases. In this way, we can find the optimum values of these morphological features in order to develop better mechanisms. Generally, there is an infinite number of solutions for a single design. However mathematics can help us obtain the optimal design, assuming there are enough constraints. Yet these constraints change according to time and environment. For example, we can find the optimum pattern of a tire in terms of friction for certain given road conditions. However the development of tire technology never stops. That means that we must properly modify our mathematical instruments to meet the demands of new conditions. Morphological Computation or 'Morphological Intelligence' Generally speaking, we can define computation as any type of calculation that uses an algorithm to produce an output based on an input. As an example, if we take a closer look at the brain even on the molecular scale, we can refer to all of the physical properties belonging to the tissue using the term "morphology." Therefore, morphology is a rather concrete term that defines the brain structure. When a human receives an input, an electrical signal arrives inside the brain and the brain generates an output signal based on its morphology. We evaluate this output signal by using the term "intelligence." Therefore, intelligence represents the convergence of an output and its optimal response. From our point of view, intelligence is just the reflection of the morphology of the brain tissue. In other words, morphology is the physical form of intelligence. The physical form of a system may include mechanical, electrical, logical, and other properties. So, to get a better understanding of morphology, we suggest dividing this term into branches. For example, the "mechanical morphology" of a system can be defined by mass, inertia, stiffness, damping, geometry, etc., and the overall mechanical performance of the mechanism can be named "mechanical intelligence." In mechanics, morphology is related to the geometry of structures and machines. Therefore, morphological computation in mechanics can be defined as the congruence between the mechanical design and its task. Mechanical, electrical, or any other types of systems have their own intelligence based on the morphological features of the design. We believe that for a given task, among various systems, simpler designs present better 102 Morphology: A Concrete Form of Intelligence morphological computations. As a result, a real dog, a smart robot dog, and a simple mechanical robot toy all use morphological computation in some way. This raises two questions: First, what is the difference between these three systems? Second, is there any computation without morphology? In our opinion, there is no computation without morphology and the difference in these three cases lies in the intelligence quotient (IQ) of the design (and more specifically, of the included morphology branch(es)). The following section aims to explain this concept in terms of wheeled and legged locomotion. Morphological Intelligence in Wheeled and Legged Vehicles According to our understanding, the objective of morphological intelligence should be to develop a system that requires a relatively smaller amount of external intelligence. Let us try to explain this in wheeled and legged vehicles. A wheel is a circular component that is intended to rotate on an axle. A wheel allows the vehicle to move in a straight, horizontal line by rolling along the contact line between the wheel and the ground. Due to the wheel's perfectly circular shape (Figure 1.a), the center of the vehicle moves on a straight horizontal line, although there are a few oscillations due to the non-rigidity of the wheel. Moving on a straight line makes wheeled vehicles quite stable and energy efficient. However, wheeled vehicles require a relatively smooth surface for locomotion, meaning that the maneuverability of wheeled machines is relatively low compared to some other mechanisms (Figures 1.b,c,d). To maximize flexibility, it is better to have a mechanism as complicated as a legged vehicle which has a larger degree of freedom of movement. However, in this case, we need additional intelligence to control the dynamics of the locomotion. It is quite difficult to design such intelligence with mechanical instruments. So, we prefer to include electronic intelligence when the complexity of the mechanical system increases. This method of design has become a rather popular trend, for electronics technology has advanced over the last century. However, as the number of motor elements in the mechanics increases, the intelligence of the electronics becomes less satisfying. Thus, this trend has begun to slow down in recent years. Instead of putting new electronics in the system while removing mechanical parts, some researchers have managed to decrease the complexity of the electronic control units by finding proper mechanical solutions for the task. They have come up with an idea for the morphological computation of mechanics [3], [? ]. Blickhan et al. [4] and Iida et al. [2] increased the intelligence of the leg mechanism by using passive elastic elements on the foot that served to decrease the complexity of the electronic intelligence. Similarly, Hauser et al. [3] and Reis et al. [1], [? ], used elastic compliant elements in a compatible shape to increase mechanical intelligence. There have also been many other efforts to increase the intelligence of the mechanism while simplifying its electrical intelligence [7], [6]. Figure 2 and Figure 3 present the simplified block diagrams of an electrically driven wheel and a bipedal legged machine. We can easily see that the voltage and displacement of the machines are common inputs and outputs, respectively, for both mechanisms. 103 Morphology: A Concrete Form of Intelligence a b c d Figure 1: Simple locomotion models. (a) wheel, (b) rimless wheel model, (c) compass gait model (d) kneed walker model. Figure 2: Simplified morphology of the intelligence of a wheeled machine. Comparing these two figures, we can see that the wheel possesses a relatively simpler morphology, although it only works on relatively smooth surfaces. Hence, to take advantage of this simple system, we must design the proper environment for the wheel. Although the legged vehicle is more maneuverable, it is much more complex than the wheel, making it difficult to control. Here, we would like to use the well-known term, "intelligence quotient," for the mechanical system in order to rate the grade of the mechanism. In locomotion, stability, maneuverability, and ease of control are three important parameters to measure the intelligence quotient (IQ) of the mechanism. The advantage of the high maneuverability of the legged vehicle makes the vehicle more difficult to control. With these considerations in mind, it can be concluded that there is, in general, an inverse correlation between ease of control and maneuverability. If the vehicle is easy to control, then it is less maneuverable; if it is highly maneuverable, it is more difficult to control. For tasks requiring high maneuverability, a simple design may not meet the desired expectations of the machine. Therefore, an additional morphological design should be used to solve this problem. Figure 3 presents a simplified block diagram of a legged vehicle with "n" number of leg joints. We can see in the figure that the incapability of the mechanics is easily addressed using electronics and logic. This point is not clear in the literature: although authors talk about the morphological computation of the intel104 Morphology: A Concrete Form of Intelligence Figure 3: Simplified morphology of the intelligence of a legged machine. ligent mechanical design, they neglect the morphological aspects of additional systems (electrical, chemical, logical etc.). In fact, when we take a closer look at these types of systems, we also see morphology. Future of the Morphological Intelligence Because both morphology and computation are general terms, a chemist, biologist, or any other researcher might use these terms as much as a mechanical engineer. We believe that a process of ramification is necessary for the discipline of morphological computation (or "morphological intelligence" as we have called it) in order to obtain more concrete applications of the term. Therefore, we want to introduce three subfields of morphological intelligence: • Morphological compatibility, • Morphological intelligence transformation, • Morphological intelligence optimization We believe that in the future, these three fields of study will guide researchers within the discipline of morphological intelligence. The biggest challenge in the computation of morphological intelligence is finding the design concept, which is what we mean by "morphological compatibility." There might be an infinitive number of solutions for the "design." Our mathematical methods need constraints to examine and find morphological features, although we do not know how it is possible to search for a design without human interference. If it becomes possible to do so in the future, we could develop inventor machines. 105 Morphology: A Concrete Form of Intelligence In morphological intelligence, for example, we transform the morphology of intelligence from mechanics to electronics or from electronics to mechanics, or we are searching for better mechanical/electronic morphological solutions. This is what we have referred to as "morphological intelligence transformation." Finding the optimum level of design parameters for a specifically defined design is relatively simple when you have strong optimization instruments. In our opinion, the improvement of morphological compatibility is related to the development of optimization techniques. We need stable, adaptive and easy-to-control engineering solutions for various tasks, but we know that these three characteristics conflict with each other in most cases. Furthermore, we encounter many other paradoxes in mechanics when increasing the complexity of our designs. By using superior optimization methods, we will be able to find better morphologies for different tasks in the future. We can examine optimal morphologies and simple engineering solutions by using and developing analytical and numerical search methods. In essence, this is "morphological intelligence optimization." Human beings can increase the level of their intelligence by learning and practicing. Human muscles and neurons perform their tasks more effectively with practice. Actually, practicing provides morphological improvement in all living organisms. However, the morphology of machines and other non-living structures are fixed, or the transformations in the mechanism are pre-defined like in a Swiss army knife or a cruise control system. In our opinion, in the future, self-transformable morphologies and self-transformable morphological intelligence will be applicable for machines, too. Since we are talking about the intelligence and morphological optimization of the mechanical systems, it is fitting also to discuss the mechanical IQ. Resultingly, we believe that in the near future a methodology to define the levels of intelligences of machines and designs will be discovered. Today, we employ complicated electronic circuits when it is difficult to design the mechanical morphology for a specific task. At the same time, however, the complexity of the electronics brings new difficulties, such as processing and transferring the data. In the future, we will manage to minimize the necessary computation for the morphological intelligence of electronics by replacing it with better mechanical morphologies. We will develop better search methods to calculate the optimal mechanical design and thus decrease the complexity of electronics [5]. There are already many optimized solutions in nature: the mechanical morphology and the "electronics" (i.e., the brain) of the body are optimized in humans as well as other complex and simple organisms. As such, we as scientists have a lot to learn from nature. The future is brimming with new disciplines that study the morphological optimization of intelligence by seeking the optimal solution through different branches of morphology. Today, mechatronics serves a similar purpose, bridging the fields of mechanics and electronics in the same way. When there is a coupling between different types of morphology, it follows that there should be different types of intelligence as well. 106 Morphology: A Concrete Form of Intelligence Bibliography [1] R. Blickhan, A. Seyfarth, H. Geyer, S. Grimmer, H. Wagner, and M. Gunther. Intelligence by Mechanics. Phil. Trans. R. Soc. A, 365:199–220, 2007. [2] H. Hauser, A. J. Ijspeert, R. M. Fuchslin, R. Pfeifer, and W. Maass. Towards a Theoretical Foundation for Morphological Computation with Compliant Bodies. Biol. Cybern., 105(5-6):355–370, 2011. [3] F. Iida, G. Gomez, and R. Pfeifer. Exploiting Body Dynamics for Controlling a Running Quadruped Robot. International Conference on Advanced Robotics, 12: 229–235, 2005. [4] R. Pfeifer and F. Iida. Morphological Computation: Connecting Body, Brain, and Environment. Biologically Inspired Approaches to Advanced Information Technology, Lecture Notes in Computer Science, 3853(2):3247–3259, 2006. [5] R. Pfeifer, M. Lungarella, and F. Iida. Self-organization, Embodiment, and Biologically Inspired Robotics. Science, 318(5853):1088–1093, 2007. [6] M. Reis, N. Maheshwari, and F. Iida. Self-organization of Robot Walking and Hopping Based on Free Vibration. International Conference on Morphological Computation, 2:91–93, 2011. [7] M. Reis, X. Yu, N. Maheshwari, and F. Iida. Morphological Computation of Multigaited Robot Locomotion Based on Free Vibration. Artificial Life, 19(1):97–114, 2013. 107 Abstract: This paper discusses morphological computation from the perspective of self-assembly. Ultimately, organic molecules are "computational machines" that mechanically conduct logical operations and achieve massive structure constructions. Their capability to realize global functionalities by locally interacting with other molecules through their body may endow artificial products with a power of vitality and higher intelligence. As biological molecules attain some fundamental vital activities, such as development or self-repair, we believe that the challenge on filling the gap between artificial machines and organic machines will provide us with hints on what it means to be a living system. Molecules and Robots Proteins form complex structures through interaction. Unlike most robots, they attain fertile representations without an explicit central coordinator, and somehow make up for the lack of computational processors or silicon-based memories through embodiment. Focusing on the tactics in such a system, in recent decades, the field of DNA-tile self-assembly has shown remarkable achievements in fabricating nano-scale structures employing molecules in a bottom-up fashion [1, 2]. The strategy of these structures is to guarantee the same environmental conditions as in molecular assembly namely, weak interaction, thermal agitation, and nucleation but artificially design the tiles with nucleotides such that they attain the desired basic formation patterns or sometimes even functions. The achievement provides the system with a tool to encode molecules for the realization of artificial synthesis at the molecular scale, in the same way that nature operates. One big question remains: "Is mimicking nature's strategy using natural materials at the natural scale the only way to realize organic activity?" So far, no one has satisfactorily answered the answer. Yes, we do see some self-organization at a larger scale; floating fallen leaves gather together and form a pattern. But no, the way they assemble is rather static and the equilibrium state is, so to speak, "dead." So, then, what is life? Or, what are the activities that define living states? Distributed entities that self-repair? Dissipative materials that show robustness in their environments? Or something that consumes negative-entropy? With a few exceptions, it seems intuitive to distinguish living entities from non-living entities. Alas, those entities must fit into the classical category of "life," which we mostly find in high school biology textbooks. Tackling the interface between life and non-life must be a thrilling research activity for some synthetic biologists, regardless of the fact that the complexity gap is still noteworthy. Researchers on "artificial life" have shown many insightful discoveries regarding what life could be, but nevertheless have been told that they missed a link to a physically plausible presence. The option of shifting to synthetic biology is yet within the discussion. In terms of high-level functionality, a series of self-assembling robots were generated in the 2000s that can stochastically self-replicate (with a supplement of constructive components), or determine the bonding to another robot, on an air-table, which shows a certain similarity to the basic chemical activity in an artificial way [3, 4, 5]. Probably if the developer declares such robots to comprise "life," it might be revealed that the profound theme had been solved. However, most of the people shall keep feeling that the mystery remains unsolved. Computation is a strong concept, which has been enabled by discretizing the world mainly for temporarily, and has allowed researchers to apply logics through write-in and read-out activities by an interpreter. Progress in the temporal dimension can also be expressed spatially, and this representation shows high affinity to a discretized space, such as cellular automata, or (self-assembling) tiling activities. Therefore, designating interactions between such tiles licenses one to conduct logic operations, and thus the quality of this designation level has always been centered as a major challenge in physically grounded distributed stochastic systems. I would not make a throw of the dice to bet on whether we are merely passive existences computing something "valuable" with molecules that compose our bodies. But I believe in free will, as you likely do too, and 109 Molecules and Robots we recognize that the state-of-the-art science has made great advances by avoiding this topic. magnet magnet body outline body outline shifting the magnet? changing the entire body shape? A B Figure 1: Shifting the position of the magnet vs. changing the entire body shape. I will attempt to explain this thought with a simple model of cm-sized magnetic components that interact like molecules, and assemble a structure. Figure 1 shows two different distributions of magnets, one positioned at the center of the body (A), and the other shifted toward the corner (B). In engineering, in order to shift the position of the magnet to endow the body with different characteristics on interacting with other magnetized bodies, implementing an actuator arises as the first solution. However, this is not trivial, neither at the cm scale nor the micrometer scale. First, it requires an energy input, which comes with the requirement of a battery, and second, the force that an actuator can exert is usually limited. Nature, on the other hand, often employs a different approach, namely recovering the initial condition using entropy, such as the proton-motive force (Figure 2). Gaining a different state by entropy has been explored in a colloidal field. In engineering, our recent results [6] suggest an interesting possibility to employ entropic (more accurately, depletion) force at larger scales. In the work, we showed that vibrating floating tiles segregate themselves from passive tiles, and form a cluster. However, no matter how well the molecules regain their initial conditions employing entropy, the fact that they passively react does not change. We are still dealing with passive existence. Having entered the century of the fastest scientific advancements ever, our knowledge of living systems on the constituents, structure, amount, and temporal dynamics is increasing every day. The discoveries have enabled us to develop a better quality of life with fewer concerns. But for this reason, I believe we should not slow down the pace of pursuing the ultimate question: who are we? 110 Molecules and Robots state A state B using enthalpy using entropy Figure 2: Nature often employs enthalpy for forward reactions while entropy for backward reaction. Bibliography [1] E. Winfree, F. Liu, L.A. Wenzler, and N.C. Seeman. Design and self-assembly of two-dimensional DNA crystals. Nature, 394(6693):539 – 544, 1998. [2] P.W. K. Rothemund. Folding DNA to create nanoscale shapes and patterns. Nature, 440(7082):297–302, 2006. [3] P. White, K. Kopanski, and H. Lipson. Stochastic self-reconfigurable cellular robotics. In IEEE International Conference on Robotics and Automation (ICRA), pages 2888– 2893, 2004. [4] S. Griffith. Growing Machines. PhD thesis, MIT, 2004. [5] N. Napp, S. Burden, and E. Klavins. The statistical dynamics of programmed selfassembly. In IEEE International Conference on Robotics and Automation (ICRA), pages 1469–1476, 2006. [6] K. Nakajima, A. M. Tientcheu Ngouabeu, S. Miyashita, M. Göldi, R.M. Füchslin, and R. Pfeifer. Morphology-induced collective behaviors: Dynamic pattern formation in water floating elements. PLoS ONE, 7(6):e37805, 2012. 111 Abstract: There has been a growing interest in the idea and conception of morphological computation and how they may bring benefits to robotics research. This manuscript aims to discuss them based on the movement mechanism of the simplest creatures: bacteria. To be more specific, we will discuss the concept of what morphological computation is by describing the way bacteria connect their "brain," body, and environment while performing a chemotaxis behavior or random walk. In conjunction with the discussion, we will also suggest some challenges and the necessary future works related with the idea of morphological computation. Morphological Computation: A Perspective Based on Bacterial Movement Morphological Computation, Morphological Control and Bacterial Movement The notion of morphological computation is rooted in the concept of embodiment, argued to have significant importance in realizing adaptivity, autonomy and learning capability in robotic system [12, 10]. Trivially means "intelligence requires body," the concept of embodiment suggests more important implications, concerned with the relation between physical and neural or control processes. In relaton with this, morphological computation is essentially about connecting the body, brain and environment [11]. The term "morphology" itself refers to the form and structure of the robot, such as its shape, size and material properties. Several concepts have been proposed in previous research to explain the notion of morphological computation. For instance, it has been proposed that morphological computation essentially means a computation obtained through interactions of physical form [2]. It has also been argued that a process can be called a morphological computation if: (1) it converts a reproducible input into a reproducible output (2) it is programmable in the sense that map between input and output is parametrized in such a way that a wide of variety of outputs can be produced (3) it has a sort of teleological embedding [3]. In [3], an interesting argument is pointed out about the difference between the term morphological computation and control. While computation refers to a mapping between output and input, whereby the input is completely given at the start of the process, the term control refers to a process generating a stream of output signals that determine reactions in the input signals, which may not be completely known at the beginning of the process. Unlike conventional control that tries to minimize the influence of robot's morphology, it is argued therein that morphological control attempts to take advantage of the morphology, exploiting the dynamics of the system for the control task. Therefore, the necessary representation for the controller does not neccesarily reflect the complete robot's state. Instead, the dynamical system should be considered as a parameterized attractor landscape, where the task of the controller is to move the system from one basin of attraction into another one, without coordinating the details of the movement pattern. The last, but not least, interesting argument described in [3] is the necessity to include stochatic processes in the framework of morphological computation, due to possible stochasticity involved in an embodied processes. In this manuscript, the concept of morphological computation will be discussed by observing the mechanism involved in the movement of the simplest creatures: bacteria, to be more exact the Escherichia coli. The concept will be derived by understanding the way bacteria's "brain", body, and their environment are connected with each other. The movement of bacteria is known as bacterial chemotaxis, due to its purpose to find sources of chemical gradient useful for their survival. The concept of bacterial chemotaxis is shown by Figure 1. Bacteria only have a single sensor with a simple mechanism that only enables them to detect the spatial chemical gradient by simply comparing current and previous value. Bacteria's actuator, namely flagellum, acts like a propeller with a one degree of freedom (1-DOF) motor [16]. 113 Morphological Computation: A Perspective Based on Bacterial Movement Figure 1: The bacterial chemotaxis mechanism. The right figure is adapted from [7] with permission. The left figure of Figure 1 shows the basic architecture of chemotaxis system in Escherichia coli. Stimulus molecules diffuse through the outer membrane of the sensor located in the front part of bacteria, and interact with the respective receptors. Due to this interaction, ultimately, all chemotaxis signals can be integrated as the level of phosporylation of the response regulator protein, commonly named as CheY. The phosporylated CheY, named as CheY-P, diffuses from the kinase of the receptors to the flagellar motor, promotes a clockwise flagellar rotation (CW) that causes the bacteria to enter a tumble mode, or changing orientation randomly. Another protein, commonly named as CheZ, has the role to increase the rate of CheY dephosporylation. The decreasing of CheY-P promotes counterclockwise (CCW) flagellar rotation results in smooth swimming episodes, the swimming mode. Due to this mechanism, it can be modeled that the concentration of CheY-P, is fluctuating, which leads to fluctuation of the probability to do swimming or tumbling mode [8, 15, 1]. The effect of this mechanism can be further seen from the top-right figure of Figure 1. Bacteria, such as Escherichia coli, have several flagella (typically 4-10) [6], where each flagellum has a tiny rotary motor at its base. However, the motions of all flagella are synchronized, which means that a bacteria can only control the flagella as a 1DOF actuator. The flagella can rotate in two ways as shown by the figure. In the swimming mode, a counter-clockwise rotation aligns the flagella into a single rotating bundle, causing the bacterium to swim in a straight line (A). In the tumbling mode, a clockwise rotation breaks the flagella bundle apart so that each flagellum points in a different direction. This allows the bacterium to efficiently receive a random force generated by collisions with surrounding water molecules undergoing Brownian motion, 114 Morphological Computation: A Perspective Based on Bacterial Movement causing the bacterium to tumble in place (B). Further study has shown that various morphological aspects in different kinds of bacteria affect their experienced hydrodynamic and swimming efficiency; namely the number of helical turns of flagellum, the helix pitch angle, flagellum length and cell body aspect ratio [13]. Based on the simple explained mechanism, bacteria are able to perform chemotaxis: a biased random walk towards a higher concentration of preferred chemical concentration to approach the source, as shown by the bottom-right figure of Figure 1. When they are placed in a uniform environment, bacteria will simply do random walk. Amazingly, in a uniform environment, the probability to switch between the tumbling and swimming behavior has been suggested to cause an emergence of a specialized random walk, known as Levy walk. In an area where the targets are sparsely distributed, it has been shown that Levy walk is more efficient than a common Brownian walk, meaning that the probability to encounter a target within a certain traveled distance is higher. This demonstrates that based on a simple mechanism, bacterial movement is not only adaptive, but also efficient. Here, the interesting question is whether it is possible to derive some concepts about morphological computation by taking a lesson from bacterial chemotaxis mechanism. Discussion Due to the involvement of stochasticity in bacterial chemotaxis, approaches to model the underlying mechansim is commonly based on stochastic differential equations [8, 15]. To discuss the way morphological computation being performed by bacteria, here we use a particular version of stochastic differential equation which is able to explain both the chemotaxis toward chemical sources and the Levy walk behavior of bacteria [8]. The equations are built based on the biological fluctuation framework, a certain perspective to describe noise utilizing phenomena in biological systems shown by Equation 1 [5, 17]. Here, x (t) and f (x (t)) are the state and the dynamics of the model at time t, with f (x (t)) designed to have some attractors in potential U (x (t)). The noise term is represented by "(t), while A(t) is a variable called the "activity," which indicates the fitness of the state to the environment. From the equation, f (x (t))A(t) becomes dominant when the activity is large, and the state transition becomes deterministic. In other words, the potential function is modified, entraining the state into a particular attractor. When the activity is small, "(t) becomes dominant, the state transition becomes more stochastic. The activity should therefore be designed to be large when the state is suited to the environment and vice versa. The complete equations that describe baceterial movement based on biological fluctuation are shown by Equations 1 3 ẋ(t) =  dU(x(t)) dt A(t) + "(t) = f(x(t))A(t) + "(t) (1) U(t) = (x(t)  h)2 (2) P (t) = exp( x(t)) (3) 115 Morphological Computation: A Perspective Based on Bacterial Movement where P(t) is the probability to tumbling (switching from swimming mode to tumbling mode). In this case, x (t) is a variable that represents the dynamics of CheY-P, and P(t) can modeled as an exponential function of x (t). Furthermore, U (t) can be modeled a potential function with single attractor h that shows a particular tumbling probability. From Equation 1, f (x (t))A(t) becomes dominant when the activity is large. In other words, the tumbling probablity is entrained into the attractor. When the activity is small, "(t) becomes dominant, the state transition becomes more stochastic, causing a stochastic fluctuation of P(t). Therefore, to model the bacterial chemotaxis, the activity should be considered as a function of the detected chemical gradient. While the mathematical model can be shown by Equations 1-3, physically the decision made by the "brain" of the bacteria depends on the diffusion and fluctuation of chemical protein CheY-P that diffuses from the sensori receptor to bacterial motors. In the above equations, it is shown that the mechanism can be modeled as dynamical system with attractor(s) (other works also model it as dynamical system with two, and even an imaginary attractor in front of the bacteria [15, 9]). When the dynamical system is entrained to the tumbling attractor, the hydrodynamic interaction between the flagella and the water molecules takes over the role of the "brain" to orient the bacteria. In turn, the change of the detected chemical concentration will lead to a change of the dynamics of CheY-P inside bacterial body, switching back its mode to a swimming mode. The alternation between two modes will lead to a chemotaxis behavior, or Levy walk as efficient random walk in a uniform environment. It is interesting to notice that here the controller's task is simply to move the system to suitable basin of attractions without coordinating the details of the movement pattern, and that the system is taking advantage of interaction of physical form involving bacteria's morphology. In summary, derived from bacterial movement mechanism, our perspectives of morphological computation, are are as follows: (1) it is a computation obtained through interactions of physical form to achieve adaptive, or at least purposeful, behavior (2) it enables morphological control; meaning that the interactions of the physical form enables the dynamics of the system to move from one basin of attraction into another without coordinating the details of the movement pattern (3) it may take advantage of stochastic processes. The importance of the concept of morphological computation in stochastic and deterministic settings has also been mentioned in [3]. Here, we essentially describe the way stochasticity may help the search for suitable attractors involving interactions of physical forms. As future work, it is an important step to understand the design principle such that they may take advantage of each other. For example, it is interesting to investigate the proper morphology of flagella under particular size of noise resulting from the bombardment of the random water molecules to timely reorient bacteria toward the gradient sources. It is interesting to mention that a control scheme based on biological fluctuation (i.e. Equation 1) has also been implemented in a more complex system: a complex humanlike robotic arm imitating the anatomy of human upper limb [14, 4]. The robot has 30 pneumatic actuators, which forms 26 muscles, such that the effects of individual actuators are unclear. Here, the noise helps to search for a suitable combination of which and how 116 Morphological Computation: A Perspective Based on Bacterial Movement strong each muscles must be contracted at each time step. It has been shown that this approach enables the robotic arm to accomplish some simple tasks like reaching a goal or imitating circular motion. However, due to the stochasticity of the movement, the approach may not be suitable for a safe human robot interaction, for example. Another challenge is the difficulty to let the complex robot arm to perform a complex task, as the search for suitable attractors is being conducted by a noise dominated behavior. These challenges indicate the identification of the functional primitives formed by a complex morphology as another important research direction. Bibliography [1] J. Armitage, I. Holland, U. Jenal, and K. B. Neural networks in bacteria: making connections. Journal of Bacteriology, 187(1):26–36, 2005. [2] P. Chandana. Morphological computation: A basis for the analysis of morphology and control requirement. Robotics and Autonomous Systems, 54:619–630, 2006. [3] R. Füchslin, H. H., R. B., L. R.H., and S. S. Morphological control: Applications on different scales exploiting classical and statistical mechanics. In The Third International Conf. on Morphological Computation, ICMC 2011, pages 42–44, 2011. [4] I. Fukuyori, Y. Nakamura, Y. Matsumoto, and S. Ishiguro. Flexible control mechanism for multi-dof robotic arm based on biological fluctuation. In The Tenth International Conf. on Simulation of Adaptive Behavior, SAB 2008, pages 22–31, 2008. [5] A. Kashiwagi, I. Urabe, K. Kaneko, and T. Yomo. Adaptive response of a gene network to environmental changes by fitness induced attractor selection. PLoS ONE, 1(1):e49, 2006. [6] R. Macnab. Escherichia coli and salmonella: cellular and molecular biology. American Society for Microbiology, 2:123–145, 1996. [7] S. Nurzaman, Y. Matsumoto, Y. Nakamura, K. Shirai, S. Koizumi, and H. Ishiguro. An adaptive switching behavior between Levy to Brownian random search in a mobile robot based on biological fluctuation. In Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1927–1934, 2010. [8] S. Nurzaman, Y. Matsumoto, Y. Nakamura, K. Shirai, S. Koizumi, and H. Ishiguro. From Levy to Brownian: A computational model based on biological fluctuation. PLoS ONE, 6(2):e16168, 2011. [9] S. Nurzaman, Y. Matsumoto, Y. Nakamura, K. Shirai, S. Koizumi, and H. Ishiguro. Bacteria-inspired underactuated mobile robot based on a biological fluctuation. Adaptive Behavior, 20(4):225–236, 2012. [10] R. Pfeifer and J. Bongard. How the body shapes the way we think: A new view of intelligence. MIT Press, 2006. 117 Morphological Computation: A Perspective Based on Bacterial Movement [11] R. Pfeifer and F. Iida. Morphological computation: Connecting body, brain and environment. Japanese Scientific Monthy, 58(2):48–54, 2005. [12] R. Pfeifer, M. Lungarella, and F. Iida. Self-organization, embodiment, and biologically inspired robotics. Science, 318(5853):1008–1093, 2007. [13] H. Shum, E. Gaffney, and D. Smith. Modelling bacterial behavior close to a noslip plane boundary: the influence of bacterial geometry. Proc. R. Soc. A, 466: 1725–1748, 2010. [14] A. Sugahara, Y. Nakamura, I. Fukuyori, Y. Matsumoto, and H. Ishiguro. Generating circular motion of a human-like robotic arm using attractor selection model. Journal of Robotics and Mechatronics, 22(3):315–321, 2010. [15] Y. Tu and G. Grinstein. How white noise generates power-law switching in bacterial flagellar motors. Physical Review Letters, 94:208101, 2005. [16] D. Webre, P. Wolanin, and J. Stock. Bacterial chemotaxis. Current Biology, 13(2): R47–R49, 2003. [17] T. Yanagida, M. Ueda, T. Murata, S. Esaki, and Y. Ishii. Brownian motion, fluctuation and life. BioSystems, 88:228–242, 2007. 118 Abstract: Morphological computation is a powerful design concept for coordinating forces from actuators of robots and their environment using their body dynamics (e.g., its geometric structure and viscoelastic distribution) to generate adaptive behavior. However, these forces are invisible to robot designers and therefore very difficult to handle, especially with an external force disturbance. To alleviate this, we focused on the plasmodium of true slime mold as a platform to discuss physical interaction between body parts, because the force balance between its body parts is considered protoplasmic streaming. By exploiting the physical interaction (or socalled morphological computation) between the body parts, the plasmodium exhibits versatile and adaptive behaviors. Inspired by this, we design a hydrostatically interacting mechanosensory oscillators. The numerical experiments show that the model produces situation-dependent oscillatory patterns with two modules, and versatile oscillatory patterns with three modules without changing any parameters of the model during the simulation run. The results indicate that embedding morphological computation as a part of control system of robots can contribute to their versatile and adaptive behaviors. Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators Introduction Morphological computation is a key design concept for building adaptive and robust robotic systems for applications to dynamically changing environments. The term "morphological computation" was coined by Chandana Paul [2] and refers to the performance of certain computational processes by a mechanical system (i.e., the body) that would otherwise have to be performed by a control system (i.e., the brain and nervous system) [3]. Given the fact that animals have a limited number of neurons (especially primitive or small organisms), they are able to generate adaptive and robust behaviors by exploiting its morphology, such as its geometric structure and viscoelastic distribution, as a computational resource. When we try to implement the concept of morphological computation into robotic systems, how to generate force distribution on the body is very important to achieve the desired functionalities (e.g., competitive forces around a joint to achieve high stiffness on the joint or unifying forces from several actuators to one direction to produce a high movement speed and a large torque). However, because such forces are invisible to robot designers, they are very hard to handle, especially when there are unpredictably changing forces from the environment. There is a simple, yet very suitable, living organism that can be investigated to tackle this challenging problem: the plasmodium of true slime mold (Physarum polycephalum, see Figure 1). In the plasmodium, the force balance between the body parts can literally be seen by protoplasmic streaming (i.e., cytoplasmic streaming). The plasmodium is a large amoebalike multi-nucleated unicellular organism, whose motion is driven by spatially distributed biochemical oscillators in its body [8]. These oscillators are hydrostatically coupled by fluid-filled tubes and induce rhythmic mechanical contractions. These lead to a pressure increase in the protoplasm, which in turn generates protoplasmic streaming according to the pressure gradient [1]. Hence, the protoplasmic streaming exactly represents the force balance between the oscillators. As a result of this morphological computation, the plasmodium exhibits amazingly versatile behaviors in a fully decentralized manner. The biochemical oscillators in the plasmodium can be modeled as homogeneous elements, where the interaction between them induces global behavior in the absence of a central nervous system or specialized organs. Yet, despite such a decentralized system, the plasmodium exhibits versatile spatiotemporal oscillatory patterns [7] and, more interestingly, spontaneously switches between these versatile oscillatory patterns [5]. Based on this behavioral diversity, the plasmodium is thought to be capable of exhibiting adaptive behavior according to the situation encountered. Since the true slime mold can be modeled as coupled oscillators as with a central pattern generator (CPG) of animals, reproducing these oscillatory patterns of the true slime mold can contribute to understanding more universal motion control on animals. In this paper, we introduce a hydrostatically coupled oscillator system inspired by the model organism, as one example of the application of morphological computation. This oscillator system consists of several homogeneous modules that are physically connected by fluid-filled tubes. The significant features of this model are twofold: (i) the force 120 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators conductance between the modules can easily be changed by tuning the fluid conductance between the modules, which leads to a change in the oscillatory patterns; (ii) three modules are capable of spontaneously generating remarkably versatile oscillatory patterns without changing any parameters of the model during the simulation run. The results show that the morphological computation allows the model to exhibit grounded behaviors according to changes in its body dynamics and versatile behaviors in a fully decentralized manner, without designing a complex interaction network between the oscillators or complex dynamics for the oscillators, such as chaos oscillators. Biological Background There are two important factors that help the plasmodium exhibit such oscillatory patterns: the phase modification of the mechanosensory oscillators and physical communication (i.e., morphological communication [4]) stemming from protoplasmic streaming. The oscillators of the plasmodium are hydrostatically coupled by tubes filled with protoplasm. By generating protoplasmic streaming through these tubes, physical interaction is induced between the oscillators. This physical interaction leads to a phase modification based on the pressure (i.e., mechanosensory information) applied by the protoplasm [14]. Hence, the versatile and adaptive behaviors in the plasmodium are attributed mainly to the synergistic effect of the phase modification and morphological communication stemming from the protoplasmic streaming. Based on the biological knowledge, we designed a mathematical model by focusing on the following points: (i) the modules are hydrostatically coupled by tubes so as to induce physical interaction between the modules, (ii) soft actuators are embedded in the modules to aid the flow of protoplasm between the modules and to act as mechanosensors based on the softness of the actuator, and (iii) phase modification exploits the mechanosensory information from these soft actuators. The details will be explained in the following section. Model Mechanical System The model consists of several homogeneous modules that are physically connected by tubes. As shown in Figure 2, each module consists of a soft outer skin, with fluid (i.e., protoplasm) inside. The outer skin consists of four mass particles and four real-time tunable springs (RTSs) used as soft actuators. The RTS is a device that we previously proposed [11, 9, 10] that can actively alter its resting length (i.e., the unstretched length of the spring). This mechanical passivity of the RTS enables the outer skin to be soft. Furthermore, altering the resting length of each RTS causes the protoplasm to be pushed and pulled competitively. In this paper, this physical interaction is simulated using a potential constraint on the area surrounded by the mass particles in each module (which will be explained in the following). As a control system, module i contains two phase 121 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators Figure 1: Plasmodium of true slime mold (Physarum polycephalum). The plasmodium of true slime mold exhibits amoeboid locomotion (with a speed of 1 cm/h) by generating thickness oscillations in its body, which are controlled in a fully decentralized manner. The white scale bar indicates 10 mm. Figure 2: Schematic of proposed hydrostatically coupled oscillator model. Diagrams of models composed of three modules (a) and one module (b). The figure has been taken from [13]. oscillators, each of which controls the resting lengths of two RTSs, RTS i,n,m (m = 0, 1), according to phase   i,n (n = v, h) (see Figure 2 (b)). Hydrostatical Coupling between Modules To design hydrostatic coupling between the modules, we simulate the protoplasm inside a module using an ideal gas. Using the ideal gas law, the pressure of module i can be 122 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators calculated as p i = N i RK V i = N i RK l i,v l i,h d , (1) where N i is the amount of substance of the ideal gas (measured in moles) inside module i, V i (= l i,v l i,h d) is the volume of module i, R is a physical constant, K is the temperature inside the module, l i,n (n = v, h) is the actual length of RTS i,n,m , and d is the thickness of the modules. Based on this equation, the potential that keeps the volume of the module constant can be written as ' i =  RKN i log V i + p ex V i , =  RKN i log l i,v l i,h d+ p ex l i,v l i,h d, (2) where p ex is the external pressure1. The first term on the right-hand side is the potential stemming from the ideal gas inside the module, and the second term is derived from the potential generated by the external pressure, p ex . As seen in the equation, the potential ' i has a concave shape along l i,v and l i,h , which provides volume conservation according to N i . This in turn induces the intra-module physical interaction and inter-module physical interactions when several modules are connected by the tubes. Therefore, the motion equation of the resting length of RTS i,n,m , l i,n , can be written as ⌘  l i,n =  T i,n   @'i @l i,n , (3) where ⌘ is the viscosity coefficient2 and T i,n is tension on RTS i,n,m . As shown in Figures 2 (a) and 3, the modules can be physically connected using the fluid-filled tubes. Through these tubes, protoplasmic streaming is generated between the modules based on the pressure gradient. This is given by  N i = X j 6=i D i,j (p j   p i ), (4) where D i,j is the fluid conductance between module i and module j. It should be noted that the total volume of the protoplasm inside all of the modules and tubes is conserved. Because of this, physical interaction is induced between the modules. Mechanosensory Soft Actuator To design a mechanosensory module, we employee an RTS, which is an elastic device that can alter its resting length and has a force sensor to sense its tension. To simplify the model, we assume that the device only expands or contracts in one dimension. This can be realized by forcibly winding/unwinding the elastic material with a mechanical 1 In this model, volume conservation is the essential characteristic, irrespective of the volume being of the gas or liquid. To simplify this, we use the ideal gas equation. The potential can be calculated from the work involved in the volume change as follows: W =   R p i dV i + R p ex dV i . 2 We assume that the variation in the resting length is slow enough to neglect the inertial force. 123 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators constraint so that it moves in only one dimension. The resting length of RTS i,n,m , lRTS i,n (  i,n ), changes according to   i,n and is given by lRTS i,n (  i,n ) =  l(1  a cos  i,n ), (5) where a is a constant in space and time (0 < a < 1) and  l denotes the mean length. Depending on its resting length, the spring stiffness of RTS i,n,m , kRTS i,n (  i,n ), varies as follows: kRTS i,n (  i,n ) = ↵ lRTS i,n (  i,n ) , (6) where ↵ is a constant given by the material and geometric properties of the elastic material. The tension on RTS i,n,m , T i,n , can be measured using the force sensor, and it is caused by the discrepancy between the actual length (l i,n ) and the resting length lRTS i,n (  i,n ): T i,n = kRTS i,n (  i,n ) * (l i,n   lRTS i,n (  i,n )). (7) It should be noted that an RTS behaves not only as an actuator but also as a (passive) spring. Control System Here, we introduce the dynamics of the control system (i.e., phase oscillator) to be implemented. The equation for the oscillator is given as [11, 9, 10]    i,n = ! n   @Ii,n @  i,n , (8) where ! n is the intrinsic frequency of the phase oscillator and the second term is the local sensory feedback, which can be calculated from the discrepancy function I i,n . This function is based on the mechanosensory information of the soft actuator, as mentioned in the prior part (Mechanosensory Soft Actuator). Note that the phase oscillators only interact with each other through the mechanical system (i.e., the protoplasm). As explained in the previous work [11, 9, 10], the plasmodium oscillators are able to sense the force from the protoplasm and tend to reduce this force by modifying their phase [14]. Based on this biological finding, we define the discrepancy function for this model as I i,n =   2 T 2 i,n , (9) where   is a coefficient that defines the strength of the feedback, and T i,n is the tension in the RTS. This function is designed to increase in value when the absolute value of T i,n increases. It should be noted that this mechanosensory information can only be produced by the mechanical softness of the actuator. Based on the discrepancy function, (8) can be rewritten as    i,n = ! n +  ↵2 l i,n lRTS i,n (  i,n )   1 ! l i,n  la sin  i,n (lRTS i,n (  i,n )) 2 . (10) 124 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators The second term on the right-hand side of (10) is the local sensory feedback that reduces the discrepancy function I i,n . It can be calculated using only locally available variables, which include the discrepancy between the controlled value and its actual value. Figure 3: Schematic of oscillator model composed of two modules. The figure has been taken from [13]. Simulation Results Behavioral Changes based on Body Dynamics (Two Modules) First, let us describe how the behavior of two modules changes according to the body dynamics. In particular, we conduct two numerical experiments: when the modules are connected by a thick tube (large fluid conductance) and when they are connected by a thin tube (small fluid conductance). The results are presented in Figure 4. In both cases, all of the phase oscillators start with an almost in-phase condition, which leads to a balance between the competitive pushing of the protoplasm (note that the phase oscillators control the resting lengths, not the actual lengths) and a high value of P i P n I i,n (total value of I i,n of the entire system). Therefore, the areas have values of 1.0, as seen in Figure 4 (a). As a result, very little of the protoplasm (gas) is exchanged between the modules. When the fluid conductance is high (D0,1 = D1,0 = 1.0), the volumes of the two modules eventually oscillate in an anti-phase manner as a result of the phase modification mechanism stemming from the local sensory feedback. This means that the two modules are exchanging protoplasm, as seen in the top plot of Figure 4(b). On the other hand, when the fluid conductance is low (D0,1 = D1,0 = 0.1), it is difficult for the two modules to exchange protoplasm. In this condition, the modules decrease the total value of the discrepancy function "inside" each module, which produces anti-phase oscillation between the two phase oscillators in each module, as shown in the top plot of Figure 4(d). These simulation results indicate that the oscillator system is capable of choosing a grounded behavior based on its body dynamics. It should also be noted that this behavioral change is produced as a result of decreasing the total value of the discrepancy function in a fully decentralized manner. We set the initial value of the area of each module as 1.0. The parameters of the model are as follows: ↵ = 5.0;   = 0.1; a = 0.2;  l = 1.0; ⌘ = 1.0; RK = 100.0; 125 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators Figure 4: Representation data of different transitions of oscillatory patterns on two modules depending on Di,j (the two direction arrow). (a) shows time evolution of the areas of the modules (top) and phase of the oscillators (bottoms) around the beginning (from 200 to 400 s) when Di,j = 1.0 and Di,j = 0.1. (b) show time evolutions of the areas of the modules (top) and phases of the oscillators (bottom) from 1200 to 1400 s when Di,j = 1.0. (c) shows the time evolution of P i P n Ii,n when Di,j = 1.0. (d) show time evolutions of the areas of the modules (top) and phases of the oscillators (bottom) from 3600 to 3800 s when Di,j = 0.1. (e) shows the time evolution of P i P n Ii,n when Di,j = 0.1. The figure has been taken from [13]. p ex = 100.0; d = 1.0; dt = 0.001; ! v = ! h = 1.0;  0,v(t = 0) = 0.001;  0,h(t = 0) = 0.0; !v = !h = 1.0;  0,v(t = 0) = 0.0;  0,h(t = 0) = 0.001;  1,v(t = 0) = 0.01;  1,h(t = 0) = 0.011; Di,j = 1.0 (when fluid conductance is high); D i,j = 0.1 (when fluid conductance is low). 126 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators 0.5 1 1.5 170050 170150 170250 170350 Time (s) A re a (a .u .) (a) Rotation mode 1 2 0 0.5 1 1.5 111050 111150 111250 111350 Time (s) A re a (a .u .) (b) Partial in-phase mode 1 2 0 0.5 1 1.5 74050 74150 74250 74350 Time (s) A re a (a .u .) (c) Partial anti-phase mode 1 2 0 0.5 1 1.5 365850 365950 366050 366150 Time (s) A re a (a .u .) 1 2 0(d) Intra-oscillation mode Module 0 Module 1 Module 2 : Phase shift : Anti-phase : In-phase Figure 5: Oscillatory patterns in three modules. These four oscillatory patterns were confirmed during one continuous simulation run without any change in the parameters. Schematic diagrams of the phase relations between three modules are shown at the upper-right corners of the plots. The relationships between two modules are indicated by =: inphase; !: 2⇡3 phase shift; and $: anti-phase. The figure has been taken from [13]. Versatile Oscillatory Patterns (Three Modules) Next, we show that a model with three modules can also generate versatile oscillatory patterns. The results are shown in Figures 5 and 6. As seen in Figure 5, we confirmed the existence of four oscillatory patterns during one consecutive simulation run without any change in the parameters: (a) a rotation mode, (b) partial in-phase mode, (c) partial anti-phase mode, and (d) intra-oscillation mode3. Furthermore, the model spontaneously switched among these four oscillation modes during one observation period without any change in the parameters, as shown in Figure 6. Figure 5 (a) shows the rotation mode, where a rotating wave is observed in the order of modules 0, 2, and 1. The phase difference between neighboring oscillators is approximately 2⇡/3. Figure 5 (b) shows the partial in-phase mode, where the volumes of two modules (0 and 2) are in-phase, and the other module (1) is in anti-phase with the first two modules. Figure 5 (c) shows the partial anti-phase mode, where the volumes of two modules (0 and 2) are in anti-phase, and module 1 barely oscillates. Figure 5 (d) shows the intra-oscillation mode, where the volumes of all of the modules barely oscillate. In this oscillation mode, two phase oscillators are in anti-phase inside each module, which 3 The names of the oscillatory patterns in (a), (b), and (c) are based on Takamatsu's work [5, 6, 7]. 127 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators 210050 210150 210250 210350 0.5 1 1.5 58050 58150 58250 58350 73750 73850 73950 74050 90850 90950 91050 91150 110050 110150 110250 110350 0.5 1 1.5 130050 130150 130250 130350 180050 180150 180250 180350 195050 195150 195250 195350 A re a (a .u .) A re a (a .u .) Module 0 Module 1 Module 2 : Phase shift : Anti-phase : In-phase 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 Figure 6: Transitions between several oscillation modes during one continuous simulation run without any change in parameters. The figure has been taken from [13]. do not require a volume change oscillation. A schematic diagram of the phase relationships between the three modules is given at the upper right corner of each plot. The relationships between two modules are indicated by !: 2⇡3 phase shift; $: anti-phase; and =: in-phase. We set the initial value of the area of each module as 1.0. The parameters of the model are as follows: ! v = 1.0; ! h = 1.004;  0,v(t = 0) = 0.0;  0,h(t = 0) = 0.01;  1,v(t = 0) = 3.14;  1,h(t = 0) = 3.15;  2,v(t = 0) = 3.16;  2,h(t = 0) = 3.17; D i,j = 1.0. The remaining parameters are the same as in the previous simulation with two modules. Conclusions We have presented a mathematical model of a hydrostatically coupled mechanosensory oscillator system that exhibits two different situation-dependent behaviors with two modules according to changes in its body dynamics (i.e., value of D i,j ), and versatile oscillatory patterns with three modules without changing any parameters of the model. The simulation results demonstrated that the oscillators can be used as a useful oscillator system, as an alternative to the previous models of a central pattern generator (CPG). One of the remarkable features of this model is that it can easily interact with external/unexpected forces-we can squish or stretch the oscillators with our hands so as to change the oscillation mode-because the oscillators can be built as real physical oscillators using cylinders and tubes [12]. Furthermore, this model can be implemented into a robot as actuators to move its body parts (such as on legged robots and snakelike robots). The rotation mode may be used for generating peristaltic locomotion of a snake-like robot (with straight arrangement of the oscillators). Partial in-phase mode 128 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators can be used for generating inching motion of a caterpillar-like robot. Alternatively, perhaps the force flows between the body parts could be designed by constructing the tube network between the oscillators to achieve the desired oscillation patterns and functionalities. Because this oscillator model can be used to visualize the force balance between the body parts and modify the force conductance between them, the model could be a powerful morphological computation tool for the design of robots. Acknowledgments The authors are deeply indebted to Ryo Idei, a former graduate student at the Research Institute of Electrical Communication, Tohoku University, and to Ryo Kobayashi, Professor of Mathematical and Life Sciences at Hiroshima University, for their valuable advice and considerable effort in the development of the robot. This research was partially supported by the Japan Science and Technology Agency (JST), CREST (JST Mathematics Program, Alliance for Breakthrough between Mathematics and Sciences (ABMS)), a Grant-in-Aid for Challenging Exploratory Research (No. 23656171), and the Tateishi Science and Technology Foundation (No. 2021005). Bibliography [1] R. Kobayashi, A. Tero, and T. Nakagaki. Mathematical model for rhythmic protoplasmic movement in the true slime mold. Journal of Mathematical Biology, 53(1): 273–286, 2006. [2] C. Paul. Morphological computation: A basis for the analysis of morphology and control requirements. Robotics and Autonomous Systems, 54(8):619–630, 2006. [3] R. Pfeifer, F. Iida, and G. Gomez. Morphological computation for adaptive behavior and cognition. International Congress Series, 1291:22–29, 2006. [4] J. A. Rieffel, F. J. Valero-Cuevas, and H. Lipson. Morphological communication: exploiting coupled dynamics in a complex mechanical structure to achieve locomotion. Journal of the Royal Society Interface the Royal Society, 7(45):613–621, 2010. [5] A. Takamatsu. Spontaneous switching among multiple spatio-temporal patterns in three-oscillator systems constructed with oscillatory cells of slime mold. Physica D: Nonlinear Phenomena, 223:180–188, 2006. [6] A. Takamatsu and T. Fujii. Spontaneous switching in coupled biological oscillator systems constructed with living cells of the true slime mold. Proc. of SICE 2004 Annual Conference, 2:1760–1763, 2004. [7] A. Takamatsu, R. Tanaka, H. Yamada, T. Nakagaki, T. Fujii, and I. Endo. Spatiotemporal symmetry in rings of coupled biological oscillators of physarum plasmodial slime mold. Physical Review Letters, 87:0781021, 2001. 129 Morphological Computation with Hydrostatic Interaction between Mechanosensory Oscillators [8] A. Tero, R. Kobayashi, and T. Nakagaki. A mathematical model for adaptive transport network in path finding by true slime mold. Journal of Theoretical Biology, 244:553–564, 2007. [9] T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. Fully decentralized control of a soft-bodied robot inspired by true slime mold. Biological Cybernetics, 102:261–269, 2010. [10] T. Umedachi, K. Takeda, T. Nakagaki, R. Kobayashi, and A. Ishiguro. A soft deformable amoeboid robot inspired by plasmodium of true slime mold. International Journal of Unconventional Computing, 7:449–462, 2011. [11] T. Umedachi, R. Idei, T. Nakagaki, R. Kobayashi, and A. Ishiguro. Fluid-filled soft-bodied amoeboid robot inspired by plasmodium of true slime mold. Advanced Robotics, 26(7):693–707, 2012. [12] T. Umedachi, R. Idei, K. Ito, and A. Ishiguro. True-slime-mould-inspired hydrostatically-coupled oscillator system exhibiting versatile behaviours. Bioinspiration & Biomimetics, 8(3):035001, 2013. [13] T. Umedachi, R. Idei, K. Ito, and A. Ishiguro. A fluid-filled soft robot that exploits spontaneous switching among versatile spatio-temporal oscillatory patterns inspired by true slime mold. Artificial Life, 19(1), 2013. doi: 10.1162/ARTL_a_00081. [14] S. Yoshiyama, M. Ishigami, A. Nakamura, and K. Kohama. Calcium wave for cytoplasmic streaming of physarum polycephalum. Cell Biology International, 34 (1):35–40, 2009. 130 Abstract: It has been argued that a robot's morphology (rather than its controller) may "compute." We hypothesize that there may be circumstances under which there is some advantage for a robot to compute using its body rather than its brain. If this is true, and if we use an evolutionary algorithm to improve the bodies and brains of robots under these circumstances, it should sometimes discover morphological computation and make use of it. Here we argue that morphological complexity may be correlated with morphological computation, and demonstrate a system in which morphological complexity evolves. We also hypothesize about how such a tool could be used to investigate how and when morphological computation is useful. Evolving Morphological Computation Complexity and Morphological Computation Imagine an artificial neural network that controls a robot. This controller computes in the sense that it transforms incoming sensor signals into outgoing motor commands. Controllers can perform more or less computation: a neural network composed of linear nodes and lacking a hidden layer can only perform linear transformations from sensing to action; networks with one or more hidden layers can perform both linear and nonlinear transformations. What about the complexity of such controllers? A common measure of complexity is entropy. Consider two neural networks with the same number of neurons and synaptic connections, but in the first network all of the synaptic weights are identical and in the second network all of the weights are different. The latter network has a higher entropy associated with it compared to the former network. Intuitively, the former network is compressible: if all of the synaptic weights are the same, each hidden node in the network will compute the same function, so the network could be replaced with a smaller network composed of one hidden neuron that computes this function. What then can be said about the relationship between computation and complexity in an artificial neural network? The exclusive or (XOR) function requires more computation than either the AND or OR function in an artificial neural network because a hidden layer is required to compute intermediate results. Also, the entropy of an ANN with a hidden layer has a higher entropy than an ANN without a hidden layer (assuming the hidden layer is employed to compute a nonlinear function). So, an ANN that computes XOR performs more computation, and has higher entropy, than an ANN that computes the AND or OR function. Although the above example does not prove that there is a correlation between the amount of computation and complexity of an artificial neural network, it does suggest, anecdotally, that such a relationship may indeed exist. Now consider the morphology of a robot. More specifically, let us consider just one aspect of its morphology: its three-dimensional shape. We can characterize the complexity of the robot's shape using shape entropy [4]. Shape entropy measures the variation in local curvature of an object: a sphere obtains a shape entropy measure of zero; a wadded-up piece of paper obtains a very large value. Indeed shape entropy seems to correlate with human designations of simpleand complexly-shaped objects [5]. We now have a common complexity measure – entropy-for both the neural network and the morphology of a given robot. If we now imagine two robots that perform the same task (such as moving over rough terrain), yet one robot has higher shape entropy and lower ANN entropy than the other robot, we could denote the former robot as performing more morphological computation than the latter robot. This approach to quantifying morphological computation bears some similarity to the work of Williams et al. [7] who showed that information theoretic measures can be used to determine whether information about the task at hand is stored in an agent's controller or in the relationship between the agent and its environment. Here however, instead of asking whether information is stored in the agent or in the environment we can measure the relative amount of information in the robot's morphology and in its controller. 132 Evolving Morphological Computation Evolution and Morphological Computation Consider now that an evolutionary algorithm is employed to improve a population of random robots such that they perform some given task. Consider further that both the three-dimensional shapes and controllers of the robots can be modified by evolution. Finally, assume that there exist multiple robots that can solve the task, but have differing degrees of morphological and control complexity. Which (if any) of these optimal robots will the evolutionary algorithm find? Which paths will evolution take? Will evolution first increase control complexity and then morphological complexity, or will it do so in the reverse order? Or will morphological complexity gradually increase over evolutionary time in step with control complexity? If a child robot is slightly more capable, morphologically more complex and simpler in terms of control than its parent robot, has evolution 'traded' control complexity for morphological complexity? In other words, has it evolved a more capable robot by taking advantage of morphological computation? The answers to these questions depend of course on many factors, including the make up of the evolutionary algorithm, the task the robots are evolved to perform and the environment in which they must do so. The Environment and Morphological Computation In recent work [1] we have explored not so much the relationship between morphological and control complexity, but rather the relationship between morphological and environmental complexity. Robots were evolved to move in two different environments. The first environment was composed of a flat, featureless high-friction plane; the other environment also contained several closely-spaced low-friction blocks of ice (see Figure 1). In order to succeed in the latter environment, robots must evolve the ability to 'reach down' into the crevices between the blocks, gain purchase on the high-friction ground below and propel themselves forward. We found that robots evolved in the icy environments exhibited higher morphological complexity than robots evolved in the flat-ground environment. One can argue that the icy environments are more complex than the flat-ground environments based on the higher Kolmogorov complexity of the icy environments: a larger program is required to define the icy environments than one required to defined the flat-ground environment. The observed relationship between morphological and environmental complexity is constrained to the kinds of robots and environments we investigated. However, this is the first work to investigate the conditions under which morphological complexity increases – or fails to increase – over evolutionary time. In future experiments one could investigate how the total amount of morphological and control complexity (and the ratio between the two) changes during evolution. The robots in both kinds of environments were controlled with fixed-complexity artificial neural networks, so it was not possible to determine whether these different environments selected for different ratios of control and morphological complexity. However in a companion paper [2] we allowed the number of mechanical degrees of freedom – and 133 Evolving Morphological Computation a b c d Figure 1: A sample of four evolved robots. a: A simple-shaped robot that evolved to locomote over flat ground. b-d: Three sample robots, more morphologically complex than the robot in (a), that evolved in the icy environments. The figures has been taken from [3].View videos of these robots here. the attendant neural control – to be modified by evolution, suggesting that in future it would be possible to investigate the relationships between morphological, control and environmental complexity. More specifically, the concept of morphological computation could be investigated in this experimental paradigm relatively easily. If the robot must travel over the tops of these icy blocks, there are two classes of robot that can succeed at this task. The first class is composed of robots that, using a complex controller that performs much computation, carefully reach down into crevices and push in the correct direction. The second class is composed of robots that have complex appendages that, through simple motions, manage to lodge in the crevices in the correct way to generate propulsion. If the evolutionary algorithm generates robots of the latter kind more often that it generates robots of the former kind, we could conclude that evolution discovers and exploits the 134 Evolving Morphological Computation concept of morphological computation for generating useful behavior. Ultimate Causation and Morphological Computation Nikolas Tinbergen made clear that in order to understand a behavioral or physiological trait of an animal, it is important to understand that trait's proximate as well as ultimate causes [6]. For example, the proximate causes of human bipedal locomotion include the various muscles, tendons and ligaments found in the leg. The ultimate cause of bipedal locomotion may be1 that natural selection favored energy-efficient walking – enabled by our particular combination of muscles, tendons and ligaments – over less energy-efficient walking. If in future work we evolve robots that exhibit morphological computation, investigating the ultimate causes that gave rise to it may shed unique light on this phenomenon. For example, we may evolve robots with complexly-shaped appendages yet simple controllers that are able to move over rough terrain. We may then find that there is a proximate cause for this robot's success: the particular curvatures of its appendages allow it to passively fit into crevices in the terrain to propel itself forward. The ultimate cause of morphological computation in this case may be that it was easier for evolution to fine-tune the appendage's curvature than it was to discover some complex controller that manoeuvered a simply-shaped appendage down into crevices. Imagine then that we were to repeat such experiments using different robots, environments and tasks and that found the evolution of morphological computation in many of them. If we then investigated the proximate causes of morphological computation in each case we might build up a general theory of how this phenomenon arises; that is, how a particular morphology performs computation. If we investigate the various ultimate causes of morphological computation in each case we might learn why this phenomenon arises: that is, how morphological computation supports adaptive behavior. Bibliography [1] J. Auerbach and J. Bongard. On the relationship between environmental and morphological complexity in evolved robots. In Proceedings of the 14th International Conference on Genetic and Evolutionary Computation, pages 521–528. ACM, 2012. [2] J. Auerbach and J. Bongard. On the relationship between environmental and mechanical complexity in evolved robots. In Proceedings of the 13th International Conference on the Synthesis and Simulation of Living Systems (ALife XIII), volume 13, pages 309–316, 2012. [3] J. E. Auerbach and J. C. Bongard. Environmental influence on the evolution of morphological complexity in machines. PLoS Computational Biology, 10(1):e1003399, 2014. 1 There are several competing hypotheses regarding the evolutionary origins of bipedal locomotion. 135 Evolving Morphological Computation [4] D. Page, A. Koschan, S. Sukumar, B. Roui-Abidi, and M. Abidi. Shape analysis algorithm based on information theory. In Proceedings of the International Conference on Image Processing, volume 1, pages I–229. IEEE, 2003. [5] S. Sukumar, D. Page, A. Koschan, and M. Abidi. Towards understanding what makes 3D objects appear simple or complex. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1–8. IEEE, 2008. [6] N. Tinbergen. On aims and methods of ethology. Zeitschrift für Tierpsychologie, 20 (4):410–433, 1963. [7] P. Williams and R. Beer. Information dynamics of evolved agents. From Animals to Animats 11, pages 38–49, 2010. 136 Abstract: The Morphological Viewpoint: a morphological computation or control system is one which is designed from a morphological point of view. A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics Introduction Casual perusal of the literature on morphological computation reveals there is no widelyaccepted formal definition of the term1 although serious progress towards a formal theory is being made [6]. There are however several features of works which appear under this label. Usually, prominence is given to the shape, form or structure of the technical systems under consideration. These systems in turn are often related to robotics, and the interaction of these robots or robotic manipulators with human beings has a prominent role. Our own field of professional activity is rehabilitation robotics and clinical applications for people with neurological impairments resulting from injury or disease. This focus further emphasises the importance of the interaction between the technical system and the human, because in this area the robotic system is usually designed to replace or augment some of the lost volitional function. It is attractive to begin the design process by having in mind the shape, form and structure of the correctly-functioning human system, and to shape the form of the technical support system to mimic or replace the parts of the human neuro-musculo-skeletal system which no longer work properly - this we might term a morphological approach. On the other hand, it is quite tempting to begin by formally specifying the functional requirements of the technical system and to proceed by building a solution which does not necessarily reflect in any direct way the human attributes of the system it is replacing or augmenting - some would say this represents a traditional engineering approach. So, when is a system "morphological" and when is it not? Turning to more authoritative sources, the Oxford English Dictionary currently defines "morphology," in a scientific context, as shape, form or external structure, especially of (a part of) an organism. From this definition, one may boldly surmise that morphological computation, or a morphological control system, has quite a lot to do with shape, form and structure and that human beings (or other living things) are closely involved in using the technical systems which emerge: shape, form and structure are emphasised at the outset; the intended function of the system plays a secondary role initially. Morphology, perhaps, can be likened to an elephant: it is hard to define, but instantly recognisable when you see it.2 Attempts to find common ground in the definition of morphological computation are reminiscent of a debate which took place in the 1950s–60s regarding adaptive control, a hot topic of that era [1]. An apparently obvious definition at the time, and one which is prominent nowadays, is that to be adaptive, a feedback controller has to adapt its parameters or structure in response to changes in the controlled system. But then it was argued that even a time-invariant controller with fixed parameters can be considered adaptive because it adjusts its output in response to changes in the measurement of the controlled system's output or in the command signal. It is quite hard to imagine the latter type of controller as being adaptive, but you have to admit that it depends on the way you think about it. In fact, the issue was neatly resolved - some would 1Just try a database search using "morphological computation." 2The "elephant test." 138 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics say sidestepped - when the proposal was put forth that an adaptive control system is one which is designed from an adaptive point of view [11]. So, you can decide yourself whether the two types of controller described are adaptive or not because it depends on how you approach the technical problem at hand. And so it might be with morphological computation and control, wherefore we propose: The Morphological Viewpoint: a morphological computation or control system is one which is designed from a morphological point of view. At a basic level, it is useful to draw here on the formal definition of "morphology" given above, viz.: pertaining to shape, form or external structure, especially of (a part of) an organism. In the sequel we will use a very simple rehabilitation robotics problem - the design and feedback control of an artificial ankle joint - to see what happens when one thinks in a morphological way (or otherwise). Rehabilitation Robotics Robotic systems intended for rehabilitation of walking, self-evidently, should come in a form that promotes and supports locomotion. It is challenging for patients with neurological impairments to walk, therefore gait rehabilitation robots have been developed to promote neurological rehabilitation, adaptation and recovery of function [10]. The physical shapes, or rather morphologies, of all dynamic systems influence their interaction with the environment [9]. Gait rehabilitation robots may initially be designed using morphological methods where effective physical shapes are determined. For example, a gait orthosis can have an exoskeleton connected by three revolute joints to mimic the lower limb. Although morphological design can lead to effective mechanical structures which allow walking, the target complex behaviours of locomotion, such as ankle plantarflexion and dorsiflexion, require engineering control [7]. Gait orthosis design thus adopts computing engineering methods based on morphological analysis and engineering control to achieve a smooth gait pattern. Robotic devices are employed clinically for rehabilitation of people with paretic limbs [4]. For the lower limbs, several medically-certified products are available; these are used for rehabilitation of walking function in patients following a stroke, incomplete spinal cord injury, or in other neurological conditions. Prominent among these devices are the Lokomat3 and G-EO4 systems (Figure 1). The Lokomat uses two rigid leg orthoses which have DC motors powering the knee and hip joints. The ankle joint is supported passively. The patient is attached to the orthoses and walks on a treadmill using the support of an overhead body-weight unloading system (Figure 1(a)). In the current version of the Lokomat, high-bandwidth feedback controls the knee and hip motors so that pre-programmed joint trajectories are followed. The operator is able to reduce the amount of guidance force so that deviation from 3Hocoma AG, Volketswil, Switzerland. www.hocoma.com 4Reha Technology AG, Olten, Switzerland. www.rehatechnology.com 139 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics (a) Lokomat. Picture: Hocoma AG, Switzerland. (b) G-EO. Picture: Reha Technology AG, Switzerland. Figure 1: Robotic devices for rehabilitation of walking. the nominal gait pattern is permitted. In this way a certain amount of compliance is introduced and the patient has to start using volitional neuromuscular inputs to maintain an acceptable gait. This facet illustrates an important point which we will examine in more detail in the sequel (Sec. ): the fact that a feedback control system for a robotics device is designed from a robust engineering perspective using high-performance DC motors does not preclude the possibility that the system can have features more readily thought to be associated with morphologically-designed components, e.g. characteristics of compliance and yielding. The G-EO's principle of operation is different: it is an end-effector system in which the patient stands on two foot platforms which are driven by DC motors (Figure 1(b)). It is thus possible to achieve planar gait as well as simulated stair climbing and descent. The possibility of compliant behaviour is more obvious in this case since the trajectories of the feet can in principle be located anywhere in 2-D space. The Lokomat and the G-EO systems both have the necessary attributes to be considered as morphological computation and control systems, but one would have to talk to the original design engineers at the Hocoma and Reha Technology companies to determine whether their design perspective was truly morphological . . . however that may be, it is certainly true that the Lokomat and the G-EO are solid engineering systems which exemplify the state of the art in rehabilitation robotics. 140 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics motor joint axis gear A B screw drive foot shank segment segment (a) Joint components. A DC motor is connected via gearing to a vertical screw drive. The drive applies a torque ⌧mj to the foot segment at connection point A. In the experiments, an external disturbance torque d is applied manually at about point B. The ankle angle ✓ is measured by an analogue encoder positioned on the joint axis. Picture: K. J. Hunt. Tmj dd Th oa m d g (b) Torques acting on the ankle joint: d is an external disturbance torque; ⌧mj is a motor-generated torque acting on the joint after translation of the motor torque ⌧m through a gearbox. The net ankle torque is ⌧ = ⌧mj + d. Figure 2: Artificial ankle joint for the Lokomat. Control of an Artificial Ankle Joint The Lokomat product as currently marketed does not have actuation of the ankle joint but our lab has developed an artificial ankle joint which integrates with the existing leg orthoses (Figure 2). The form of the joint was chosen to mimic a simplified, planar human ankle joint. It is simplified in the sense that it exhibits only planar rotational motion and that it is driven by a single DC motor and gearbox which can produce both dorsiflexion and plantarflexion. The DC motor and gear assembly is the same as that used in the knee and hip joints of the Lokomat's orthoses. The concepts under discussion will be elucidated using a simple feedback control loop we developed for the artificial ankle joint (Figure 3). Joint dynamics are often represented in the linear time-invariant form J  ✓ = ⌧ kv  ✓ ks✓, where ✓ is the angular deviation from an arbitrary neutral position and ⌧ is the net joint torque. J is the moment of inertia while ks and kv represent the joint's intrinsic stiffness and viscous damping. These 141 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics Cm Pm g Tm Tmj C P ths th d e TTms motor torque controllerjoint controller gear joint dynamics g1 Tjs Figure 3: Feedback loop for control of the artificial ankle joint. ✓ is the joint angle and ✓⇤ its setpoint/reference value; the tracking error is e = ✓⇤ ✓. The net joint torque is ⌧ while d is an external disturbance torque. The motor-torque controller is internal to the DC motor control unit: it comprises a motor controller Cm acting on the motor plant Pm to keep the motor torque ⌧m close to its reference value ⌧⇤m. ⌧mj is the motor torque referred to the joint axis via gearing ratio g and ⌧⇤mj is its effective setpoint value. dynamics are represented as the transfer function P : ⌧ ! ✓: P = 1/J s2 + kvJ s+ ks J . (1) The ankle joint is driven by a DC motor5 and gearing with ratio g which results in a torque ⌧mj acting at the joint axis. The net joint torque comprises the motor-generated component, ⌧mj , and an external disturbance torque d, i.e. ⌧ = ⌧mj + d (Figure 2(b)); the angle and the moments acting on the joint are defined to be positive in a clockwise direction. The motor torque ⌧m = ⌧mj/g is controlled to a reference torque ⌧⇤m using a feedback loop internal to the motor's control unit6 (Figure 3); the torque controller is implemented internally as a current controller. The ankle joint dynamics are modified by a linear time-invariant compensator with transfer function C(s), which forms part of the feedback loop shown in Figure 3. Input to the compensator is the error signal e which is the deviation of the angle from a reference value ✓⇤: e = ✓⇤   ✓. The compensator parameters will be determined here using simple impedance-like control strategies which aim to modify the joint's intrinsic stiffness and damping to alternative desired values. This means that, in contrast to model-based analytical control approaches, the parameters of the dynamic model P are not required for determination of the compensator parameters. We proceed from the point of view that the compensator C is to be designed to achieve compliant ankle joint behaviour. In this view, the ankle joint should yield to the external disturbance torque d. One way of characterising this is to require the joint angle to respond to the disturbance in accordance with a pre-specified impedance law given by a desired closed-loop stiffness k1 and damping k2. Considering for simplicity steadystate conditions, our goal is to achieve a compliant joint response where the pliance is 5RE40 24 V, 150 W DC motor, Maxon Motor AG, Switzerland. 6ADS_E 50/10 servo amplifier, Maxon Motor AG, Switzerland. 142 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics characterised in steady state by the stiffness k1. The joint position ✓ should then respond to a constant external disturbance torque d according to d = k1✓ , ✓ = d/k1 as t ! 1. The key transfer function which can be used to analyse the compliance (or otherwise) of the closed-loop system is that describing the relationship between d and ✓, known in control engineering circles as the load sensitivity function [2]: d ! ✓: G✓d(s) = P (s) 1 + C(s)P (s) . (2) In steady state, i.e. ! ! 0, assuming the plant to have low-pass behaviour, we have |CP (j!)|   1. From Equation (2) it follows that the steady-state angle obtained in response to a constant disturbance torque (assuming for the moment a zero reference angle) is ✓ss ⇡ lim !!0 |C(j!)| 1d. (3) It turns out therefore that the steady-state compliance of the joint is related to the inverse of the compensator gain: the stiffness is then equivalent to the steady-state compensator gain lim!!0 |C(j!)|. A key design decision from a control engineering perspective is whether or not to include integral action in the compensator. For a Type-0 plant, i.e. a plant with no intrinsic integral action, the compensator will usually be designed with an integrator to eliminate steady-state reference-tracking error: with integral action the compensator has infinite gain at zero frequency so that any steady-state uncertainty is eliminated. But this is not what we want in the design of a compliant joint since the stiffness is then infinite and the compliance zero. Turning back to our compliant way of thinking, therefore, we consider first the case when the compensator is designed as indicated above without integral action, e.g. a simple impedance controller C(s) = k1+ k2s having stiffness k1 and damping k2. In this case lim!!0 |C(j!)| = k1 and Equation (3) gives ✓ ⇡ 1k1d , d ⇡ k1✓: this reveals that the desired compliance is attained. Now we consider a compensator with integral action, e.g. C(s) = k1+k2s+ 1k3s , which results in |C(j!)| ! 1 as ! ! 0. This in turn, from Equation (3), leads to ✓ ⇡ 0 (or the neutral position if the reference angle is non-zero). In this case the constant disturbance torque is countered by a motor-generated torque which forces the joint back to the neutral position and which in steady-state has the same magnitude as the disturbance torque. This control strategy is non-compliant because the compensator, with infinite steadystate gain, will always tend to drive the tracking error to zero by forcing the joint back to the neutral position as in Test 4 below (Figure 7). One objection which might be raised at this point by a control engineer is that the compensator structures we have discussed up to now are non-proper. The compensator C(s) is in general a transfer function in the complex variable s, which can be represented as C(s) = G(s)H(s) with G and H the numerator and denominator polynomials in s. From a practical perspective, it is usually important to make sure that C is strictly proper, i.e. degG < degH. This condition ensures the compensator gain rolls off at high frequency thus protecting the loop from the effects of high-frequency measurement noise: 143 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics when C is strictly proper, lim!!1 |C(j!)| = 0. This issue can easily be resolved with the impedance controller C(s) = k1+k2s by adding to the damping term a low-pass filter with a bandwidth above the frequency of interest for closed-loop response characteristics but below the frequency range of any undesirable noise. In the situation discussed above, the desired stiffness and damping properties will be maintained in the frequency range relevant to the performance and behaviour of the joint. Experimental Results We designed a series of experiments with the artificial ankle joint in order to illustrate the concepts developed above. All experiments started at a neutral joint reference position of ✓⇤ = 30 deg. The joint was then moved as described below by the experimenter applying upward or downward forces at the end of the "foot" segment close to point B (Figure 2(a)). This manual intervention corresponds to the external disturbance torque d (Figs. 2 and 3). In the first test, the controller was designed to give a compliant pure-stiffness characteristic with stiffness k1 = 2.4 Nm/deg, no damping, k2 = 0, and no integral action, k3 ! 1 (Figure 4). The result shows that the desired pure-stiffness behaviour was achieved almost exactly (Figure 4(b)) and that the behaviour was compliant: when moved to a position of approximately 20 or 40 deg, a constant joint moment is generated and the controller makes no further attempt to force the joint back to the neutral position (Figure 4(a), lower graph). The joint stiffness can be assessed using plots of the motorgenerated joint torque against joint angle (e.g. Figure 4(b)) because in steady-state or slow-movement conditions the magnitude of the external disturbance torque applied to the joint must be approximately equal to the motor-generated torque ⌧mj . In the second test, damping was added to the controller by setting k2 = 1 Nm*s/deg (Figure 5). The damping term k2s was augmented as described above by a first-order low-pass filter with time constant 0.1 s, i.e. the filter transfer function was 10.1s+1 ; without this filter, measurement noise from the angle sensor was amplified to an unacceptable degree. There is now substantial deviation from a pure-stiffness characteristic (Figure 5(b)) but the joint behaviour remains compliant since a constant joint torque is generated at the off-neutral positions of 20 and 40 deg (Figure 5(a), lower graph). The impedance characteristic is somewhat elliptical in shape and roughly symmetric around the line of constant stiffness (Figure 5(b)). The deviation from the dashed line of constant stiffness results from the damping component k2 d✓dt generated during dynamic transitions between the two off-set-point angles of 20 and 40 deg. The third and fourth tests were carried out with k1 = 2.4, k2 = 0 and with integral action in the compensator: k3 was set to 4 deg*s/Nm. In the third test (Figure 6), the experimenter attempted to maintain a joint-angle profile similar to that used in tests 1 and 2. At the off-set-point angles of 20 and 40 deg the integral component acts on the constant set-point error and the motor-generated joint torque increases (Figure 6(a), lower graph). The experimenter had to gradually increase the torque applied manually in the opposite direction in order to match the increasing motor-generated torque and main144 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics 0 5 10 15 20 25 30 15 20 25 30 35 40 45 time [s] jo in t a n g le [ d e g ] 0 5 10 15 20 25 30 −20 0 20 time [s] jo in t to rq u e [ N m ] setpoint actual (a) Joint angle ✓ and torque ⌧mj . In the lower graph, the setpoint curve is obscured by the data points. 15 20 25 30 35 40 45 −30 −20 −10 0 10 20 30 joint angle [deg] jo in t to rq u e [ N m ] (b) Joint torque ⌧mj vs. joint angle ✓: pure stiffness characteristic of 2.4 Nm/deg (-ve slope of dashed line, partly obscured by data points). Figure 4: Test 1. Compliant behaviour with stiffness only: experimental results with k1 = 2.4 Nm/deg (stiffness), k2 = 0 (no damping) and k3 ! 1 (no integral action). The joint was manually moved between the angles of approximately 20 and 40 deg and held at these levels for a short time between moves. 0 5 10 15 20 25 30 15 20 25 30 35 40 45 time [s] jo in t a n g le [ d e g ] 0 5 10 15 20 25 30 −20 0 20 time [s] jo in t to rq u e [ N m ] setpoint actual (a) Joint angle ✓ and torque ⌧mj . 15 20 25 30 35 40 45 −30 −20 −10 0 10 20 30 joint angle [deg] jo in t to rq u e [ N m ] (b) Joint torque ⌧mj vs. joint angle ✓: clear deviation from a pure stiffness characteristic of 2.4 Nm/deg (-ve slope of dashed line), but the approximately elliptical response is roughly symmetric around the line of constant stiffness. Figure 5: Test 2. Compliant behaviour with stiffness and damping: experimental results with k1 = 2.4 Nm/deg (stiffness), k2 = 1 Nm*s/deg (damping) and k3 ! 1 (no integral action). The joint was manually moved between the angles of approximately 20 and 40 deg and held at these levels for a short time between moves. 145 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics 0 5 10 15 20 25 30 15 20 25 30 35 40 45 time [s] jo in t a n g le [ d e g ] 0 5 10 15 20 25 30 −20 0 20 time [s] jo in t to rq u e [ N m ] setpoint actual (a) Joint angle ✓ and torque ⌧mj . 15 20 25 30 35 40 45 −30 −20 −10 0 10 20 30 joint angle [deg] jo in t to rq u e [ N m ] (b) Joint torque ⌧mj vs. joint angle ✓: clear deviation from a pure stiffness characteristic of 2.4 Nm/deg (-ve slope of dashed line). Figure 6: Test 3. Non-compliant behaviour due to integral action: experimental results with k1 = 2.4 Nm/deg (stiffness), k2 = 0 (no damping) and k3 = 4 deg*s/Nm (integral action). The joint was manually moved between the angles of approximately 20 and 40 deg and held at these levels for a short time between moves despite the increasing motor-generated joint torque caused by integral action which had to be countered by the experimenter increasing his external disturbance torque d. tain the constant joint position. The impedance characteristic is substantially different from that of a stiffness (Figure 4(b)) or stiffness-damping compensator (Figure 5(b)): see Figure 6(b). This clearly demonstrates the non-compliant behaviour of the compensator with integral action. The same compensator with integral action was used in the fourth test, but the experimental strategy was changed. The joint was moved initially to an angle of approximately 40 deg. At this angle, the joint torque had a value of approximately 30 Nm (Figure 7(a), lower graph). The experimenter then attempted to keep the torque at around this value, but to achieve this he had to allow the joint to move gradually back towards the neutral position of ✓⇤ = 30 deg (Figure 7(a), upper graph). The impedance characteristic again deviates considerably from that of a pure stiffness (Figure 7(b)), thus further illustrating the non-compliant nature of a compensator with integral action. The first four tests were contrived to illustrate the concepts of compliance and noncompliance. The fifth and final test shows what happens in the more realistic situation when the reference angle ✓⇤ has a profile which is similar to the ankle-angle profile of normal walking [5]: the compliant pure-stiffness control strategy still gives accurate reference tracking (Figure 8). The behaviours we have seen can be further understood in terms of the load sensitivity functions (Eqs. (2)–(3)) by considering the Bode magnitude plots of 1/C(s) shown in Figure 9 for the three compensators used: the proportional (P) stiffness controller Cp = 2.4, the proportional-derivative (PD) stiffness-damping controller Cpd = 2.4+ s0.1s+1 and 146 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics 0 5 10 15 20 25 30 15 20 25 30 35 40 45 time [s] jo in t a n g le [ d e g ] 0 5 10 15 20 25 30 −20 0 20 time [s] jo in t to rq u e [ N m ] setpoint actual (a) Joint angle ✓ and torque ⌧mj . 15 20 25 30 35 40 45 −30 −20 −10 0 10 20 30 joint angle [deg] jo in t to rq u e [ N m ] (b) Joint torque ⌧mj vs. joint angle ✓: clear deviation from a pure stiffness characteristic of 2.4 Nm/deg (-ve slope of dashed line). Figure 7: Test 4. Non-compliant behaviour due to integral action: experimental results with k1 = 2.4 Nm/deg (stiffness), k2 = 0 (no damping) and k3 = 4 deg*s/Nm (integral action). The joint was manually moved to an angle of approximately 40 deg following which the experimenter allowed the joint to move back towards the neutral position of ✓ = 30 deg while attempting to keep the torque magnitude at a value of around 30 Nm. 0 0.5 1 1.5 2 2.5 −10 0 10 20 30 40 50 time [s] jo in t a n g le [ d e g ] setpoint actual 0 0.5 1 1.5 2 2.5 −50 0 50 time [s] jo in t to rq u e [ N m ] setpoint actual Figure 8: Test 5. Ankle trajectory control. The reference/setpoint angle ✓⇤ (dashed line, upper graph) has a profile similar to the joint-angle profile of normal walking. The lower graph shows the joint torque ⌧mj and its setpoint ⌧⇤mj. Here, k1 = 2.4 Nm/deg (stiffness), k2 = 1 Nm*s/deg (damping) and k3 ! 1 (no integral action). 147 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics the proportional-integral (PI) controller Cpi = 2.4 + 14s . For the P and PD controllers the steady-state magnitude of 1/C is -7.6 dB (= 20 log10 12.4) so that the compliant stiffness characteristic with respect to external torque is achieved. For the PI controller the steady-state magnitude of 1/C is  1 dB (= 20 log10 11): the infinite steady-state controller gain makes the behaviour non-compliant. 10 −3 10 −2 10 −1 10 0 10 1 −60 −50 −40 −30 −20 −10 0 10 frequency [rad/s] m a g n itu d e [ d B ] 1/Cp (a) 1/Cp: this controller has stiffness only, and is just a P-controller. 10 −3 10 −2 10 −1 10 0 10 1 −60 −50 −40 −30 −20 −10 0 10 frequency [rad/s] m a g n itu d e [ d B ] 1/Cpd 1/Cpi (b) 1/Cpd: controller with stiffness and damping (PD-controller); 1/Cpi: controller with proportional gain and integral action (PI-controller). Figure 9: Frequency-magnitude responses of the approximate torque disturbance transfer functions 1/C: d ! ✓: G✓d(s) = P (s)1+C(s)P (s) ⇡ 1 C(s) , Equation (2). From Equation (3), ✓ss ⇡ lim!!0 |C(j!)| 1d. 148 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics Discussion Rehabilitation robots in general and the ankle joint in particular can easily be packed in a morphological box; the problem of designing a rehabilitation robot lends itself well to the morphological way of thinking. Such devices are usually attached to, or at least used by, a human being and their form often mimics human biomechanical structures. The artificial ankle joint with its electromechanical components and its programmable control unit neatly matches the morphological control concept of farming out from a central processing unit - in this context, the human brain - computation and control structures to external materials and components. When one begins the design process in this world view the process and product can certainly be regarded as morphological. But a traditional control engineering design process may also lead to a system which, in retrospect, can be painted with the morphological brush. The ankle joint control example shows that behaviour obtained on the basis of classical engineering concepts can be reinterpreted in terms more in tune with morphology. Engineering concepts for the design of automatic control systems have been around for a very long time [3]: they can provide a solid basis for design and analysis, and they do not preclude behaviour concepts to the fore in morphology. In the end, the name given to your design process depends largely on the way you think about it: if you wish to design a morphological control system you simply start from ... The Morphological Viewpoint: a morphological computation or control system is one which is designed from a morphological point of view. Bibliography [1] K. J. Åström. Adaptive feedback control. Proc IEEE, 75(2):185–217, February 1987. [2] K. J. Åström and R. M. Murray. Feedback Systems. Princeton University Press, 2008. [3] H. W. Dickinson. James Watt: craftsman and engineer. Cambridge University Press, 2010. [4] A. Esquenazi and A. Packel. Robotic-assisted gait training and restoration. Am J Phys Med Rehabil, 91(11 Suppl 3):S217–S231, Nov 2012. doi: 10.1097/PHM. 0b013e31826bce18. [5] J. Fang, H. Gollee, S. Galen, D. B. Allan, K. J. Hunt, B. A. Conway, and A. Vuckovic. Kinematic modelling of a robotic gait device for early rehabilitation of walking. Proc Inst Mech Eng H, 225(12):1177–1187, 2011. [6] R. M. Füchslin, A. Dzyakanchuk, D. Flumini, H. Hauser, K. J. Hunt, R. H. Luchsinger, B. Reller, S. Scheidegger, and R. Walker. Morphological computation 149 A Morphological Viewpoint: Juxtaposition of Design Approaches for Locomotion-Rehabilitation Robotics and morphological control: steps towards a formal theory and applications. Artificial Life, 19(1), 2013. [7] K. J. Hunt, H. Gollee, and R.-P. Jaime. Control of paraplegic ankle joint stiffness using FES while standing. Med Eng Phys, 23:541–555, October 2001. [8] J. Martin, SJ. The Jesuit Guide to Almost Everything. Harper Collins, New York, USA, 2012. [9] R. Pfeifer, M. Lungarella, and F. Iida. Self-organization, embodiment, and biologically inspired robotics. Science, 318:1088–1093, 2007. [10] J. Stein. Robotics in rehabilitation: technology as destiny. Am J Phys Med Rehabil, 91(11 Suppl 3):S199–S203, Nov 2012. doi: 10.1097/PHM.0b013e31826bcbbd. [11] J. G. Truxal. The concept of adaptive control. In E. Mishkin and L. Braun, editors, Adaptive Control Systems, chapter 1. McGraw-Hill, New York, USA, 1961. 150 Abstract: A design perspective is taken towards understanding the field of morphological computing from a theoretical and practical perspective. A set of design principles is explored through a chemical morphological computing platform, whereby the physical properties of substances such as, oil droplets, directly inform design tactics. These are differently framed to the modes of thought that shape our thinking about machine interactions by using the perspective of process philosophy and its technical embodiment – the assemblage operating system. Specifically, the Bütschli system, which self assembles from the addition of strong alkali to an olive oil field, is considered as a programmable soft robot that can be guided using internal and external chemical cues. This model system is used to explore the metaphysical, material, design and technical challenges in designing with morphological computing where a range of potential application is also discussed ranging from the cybernetic Hylozoic Ground installation for the 2010 Venice Architecture Biennale and the architectural project Future Venice that proposes to direct the action of programmable droplets to grow an artificial reef under the city that stand on narrow wood piles. The aim being to spread its point load and stop it from sinking so quickly into the soft delta soils on which the city has been founded by transforming the stiletto heels of the current foundations into platform boots. Morphological Computing and Design Introduction In the invitation to contribute to this volume, the editors asked, as one possibility, the authors to expose their point of view to the field of morphological computation by providing answers to five questions, namely: • What do you mean by morphological computing? • Which research fields and technologies will be influenced by Morphological Computation? • Where do you see concrete applications of Morphological Computation in the near or far future? • What are the main challenges in order to turn Morphological Computation from an idea into a field of which institutes will be named after? • How can Morphological Computation facilitate the interaction of humans and machines, both in handling and programming? In this essay, I will present my perspective. What to you Mean by Morphological Computation Matter at equilibrium is "blind," but far from equilibrium it begins to "see". Ilya Prigogine in [1]. Morphological computing is a term that has originated from the field of robotics [2] where the body of the robot contributes to the performance of the system. Morphological computing has particular relevance in soft robotics where materials with complex, nonlinear properties such as, elastomers [4], or oil droplets [5], [6], are incorporated into, or embody, the robot. Morphological computing applications span the Two Cultures [7]. Scientific research groups exploring these principles include Martin Hanczyc and Sherif Mansy at the University of Trento, Lee Cronin, at the University of Glasgow, Klaus-Peter Zauner at the University of Southampton, Gabriel Villar at the University of Oxford, the Artificial Intelligence Laboratory in the Department of Informatics at the University of Zurich, and Andy Adamatzky at the University of West England. Design-led practices include the architectural research group AVATAR (Advanced Virtual And Technological Architectural Research) at the University of Greenwich, Martyn Dade Robertson at the University of Newcastle and the Dutch generative artists Erwin Driessens and Maria Verstappen, who use a range of material processes to evolve images and sculptures. Such practices work with a range of principles and techniques that could be considered as various forms of morphological computing with qualitatively different outputs to other forms of computation such as, digital computing that support our current technologies of "making." Morphological computing may be distinguished from a range of computing practices by its shared ontology with the field of robotics and by the deliberate use of the physical 152 Morphological Computing and Design Figure 1: Bütschli droplet, an example of a soft robot that does not possess any mechanical components. While oil molecules form the bulk of droplet and therefore act as a possible carrier system for fat-soluble compounds, the interface at the oil/water surface of the droplet enables the droplet body to respond to chemical changes in its external or internal state. This opens us the possibility of directing the droplet body by applying internal and external programs that are encoded in chemistry. For example, chemotaxis may be used to shape autonomous movement expressed by this simple chemical system, whereby the soft robot will move rapidly towards a chemical "food" source [5]. properties of a morphological computing system into a design or engineering solution. Unlike "natural" computing, a term that has been informed by Alan Turing's interest in the computational powers of Nature and broadly refers to computing practices that examine the capabilities of natural organisms using a spectrum of platforms to better understand and reflect the properties of living things such as, adaptation, learning, evolution, growth [8], morphological computing is not primarily "inspired" by natural systems. It also differs to unconventional computation, which is also a spectrum of broadly defined practices that have been described from a range of perspectives including, technical, logical, scientific-theoretical and philosophical [9], [10] that "have been only recently invented, operate with some exotic principles, and ... have not been yet introduced on the market" [11], as it has an overreaching set of principles, rather than being grouped according to a loosely associated set of shared characteristics. Proponents of unconventional computing propose the ambitions of the discipline are to go beyond the standard models of computing such as, the Turing machine and von Neumann architectures, which have dominated computer science for more than half a century and therefore proponents are adopting a counterpoint position, rather than proposing a specific theory of computation [12], [13]. For example, slime mold computing, has been used to identify the shortest route in compound pathways [14] and dynamic chemistries can process complex information [15], [16], yet the similarities between these forms of computation are not programmatic but aesthetic. Conversely then, morphological computing possesses a set of discrete operations that are framed by the science of complexity, the phenomena associated with process philosophy, which is a metaphysical worldview in 153 Morphological Computing and Design a continual state of flux and therefore concerned with the idea of "flow" as a primary quality of experience [17] that engages matter at non-equilibrium states. It may also be regarded as a technological and philosophical successor to the field of cybernetics where it transgresses traditional materials and begins to intersect with the chemical and biological realms through a constructivist, or "synthetic," exploration of material performance, which deals with possible outcomes that occur within the limits of a system, not the certitudes associated with classical science. Which Research Fields and Technologies will be Influenced by Morphological Computation? Morphological computing has the potential to influence a broad range of design practices that are concerned with biological and ecosystem design, for although it is at a very early stage of development, it offers an exciting new set of tools that may help us develop ways of working that move us on from industrial mores and practices, towards ecological ones. Morphological computing techniques are pertinent to the way that we currently think about the natural world, as being embodied, they are responsive to environmental influences. This strategically creates opportunities to apply new principles in the shaping of our living spaces. There is a global need for methods of production that promote healthy ecological relationships between our communities and the environment, which do not damage our habitats but physically integrate our buildings into our ecosystems in ways so they may contribute to the health of a site through lifelike qualities such as self-repair, propagation, movement and decomposition. Yet, these toolsets do not currently exist in design practice. Currently the practice of biomimicry occupies this desired design space by emulating the forms and functions of the natural world. Yet, these approaches are constrained by being implemented through industrial methods of production and therefore share the same damaging outputs of modern buildings and operate according to the same resource consumption paradigms [18]. Potentially, morphological computing techniques enable designers to change the expectations of making a product or a building, where the production process does not inevitably damage its surroundings but may positively contribute to local ecosystems. While the emerging field of Bio Design proposes to go "beyond" biomimicry and is experimenting with bioprocesses to venture beyond formal representations of the natural world [19]. It is at its earliest stages of development and does not have a formal method of production. Rather, Bio Design adopts an experimental and speculative approach towards its design propositions, where there are plenty of opportunities for designers to explore morphological computing approaches. For example, Magnus Larssen's project Dune (2010) proposes an architectural-scale, bacterial sandstone printing system, which can fix sand particles using the metabolic properties of bacteria and thereby begin to reverse the process of desertification [20]. Larssen's early laboratory prototypes of working with organisms lend themselves to morphological computing techniques where the synthetic abilities of the bacteria feed back into the structure of the sandstone. A much older yet similar system exists at the Mother Shipton caves in Yorkshire, England where heav154 Morphological Computing and Design ily mineralized waters are directed through suspended soft objects that become petrified in a matter of weeks and are exhibited in the local museum Figure 2. By philosophically Figure 2: Soft toys are suspended in heavily mineralized natural waters and become petrified through natural computing processes. decoupling the practice of morphological computing from industrial ambitions designers may explore new conceptual spaces and production processes that do not need to work at high speeds but may be phased with ecological rhythms. Such an approach may help designers develop new, sustained, material relationships between human settlements and their local environments to produce prolonged effects that evolve over different timescales. Morphological computing techniques, methods and technologies could therefore enable practitioners of the built environment for example, to go beyond prioritizing the qualities of the old models of urban development, which are dictated by industrial methods of production that involve the secondary distribution of resource and energy supplies through inert objects that we call buildings. Instead, designers may embrace unfamiliar new qualities in their aspirations and designs that transform the way we inhabit our living spaces that promote new kinds of activity such as, increasing fertility, promoting biodiversity and even healing our torn ecosystems. Yet, changes in the infrastructure of our living spaces are needed if such eventualities are to be fully realised. For example, new kinds of systems, such as bioprocess-enabled architectural "organs" could perform a range of metabolic functions within a space. These designed sites of activity may produce heat, filter water or fix carbon dioxide. Such systems could be invisible to the inhabitants by occupying under-imagined sites within our buildings such as, under floors – but they could also be highly visible and exist as fetishized objects such as, in Phillips Microbial home [21], where bio processors are situated within voluptuous shapes as their interior processes transform waste products into useful substances that are then exchange and transformed through a locally defined ecology. Strategically positioned, these architectural organs may give rise to buildings with physiologies that strengthen the material exchanges within a community through networks of metabolic processes and act as biotic, life promoting oases for human and nonhuman communities. Governance of these systems may be managed by a combina155 Morphological Computing and Design tion of different computing types from digital computing through embedded microfluidic systems that regulate flow of resources in the system, it is also likely that forms of morphological computing will also be used in semi-autonomous regulatory feedback systems for example, where the density of microalgae within a volumetric space limits its growth. Where Do You See Concrete Applications of Morphological Computation in the Near or Far Future? Morphological computing is extremely relevant to a range of design practices from architecture to fashion and product design that are looking for new production platforms with ecologically beneficial impacts, which enable them to imagine and work with materials in new ways. Morphological computing offers designers an opportunity to work at a deeper level of design than those approaches that focus on the nature of a final product. It allows practitioners to shape outcomes that precede the emergence of form by shaping connections that take place between design units through designed networks of interaction. This facilitates the evolution of material systems at far-from-equilibrium states to operate under their own momentum and therefore becomes co-designers of the outputs. Such an approach distributes agency within design practice and contrasts with modern industrial practices that adopt a top-down approach to the manufacturing of an object, where morphologies are pre-determined and the preferred materials are inert. For example, artists Erwin Driessens and Maria Verstappen evolve images and sculptures using a range of technologies that incorporate the physicality of the system into the output such as, Sand Box (2009), a diorama in which a sand bed is continuously transformed by means of wind. Audiences can observe the continual transformations of a sand and wind based system through a small window and observe a glimpse of a world where another climate prevails [22]. The agency that is giving rise to these transformations does not reside in any one component of an installation but also resides within the chemistry of the transmuting materials and the environmental context in which they are situated. Since morphological computing produces a qualitatively different set of outputs than industrial forms of design, it requires a unique language to frame expectations. While the language of morphological computing itself is based in the common language of physics and chemistry, which is shared by all natural systems, the linguistics that describe its operating systems can be drawn from process philosophy [17]. It shares a set of principles that view the reality as an condensation of physical, organic, social, and cognitive processes, which operate across many levels of organization and includes thinkers such as, Heraclitus, Gottfried Wilhelm Leibniz, Georg Hegel, Friedrich Nietzsche, Martin Heidegger, Jacques Derrida, Alfred North Whitehead, Henri Bergson, William James and John Dewey, although this list is not exhaustive. In classical Western philosophy the world is imagined as being made up of objects, whose operating system is the "machine." In contrast, the operating system of process philosophy is based on the concept of an "assemblage," which can deal with constant 156 Morphological Computing and Design change. The concept of assemblage is from the French word "agencement" used by Gilles Deleuze and Felix Guattari [23] to denote specific connections between groupings of actants, which are empowered bodies [24]. Rather than forming hierarchies of order, as in the operating system of machines, assemblages form loose, reversible associations with each other [25]. Yet, unlike machines they are capable of radical acts of transformation as they reach tipping points in their order, which may give rise to new meaning or effects [26]. Yet because assemblages are sensitive to and can respond to changing internal, or external conditions, they may also be viewed as a form of technology that can be shaped by morphological computing techniques and may even be considered as a Nature-based production platform since natural systems are also in a continual state of flux. My design research operationalizes the concept of assemblage to identify the materials, technologies and infrastructures that enable designers to develop their own approach to morphological computing. Assemblages do not yet exist as mainstream technology and are not formalized in terms of their engineering, operations or outputs. Yet, these systems are relevant to design and engineering practices, since they offer the potential to construct spatial programs and realize them in different ways to machine-based paradigms. In response to this opportunity, I have developed a model morphological computing platform using a lifelike chemical system, which was first described by Otto Bütschli in 1892 [27]. The Bütschli system is an emergent soft, wet, chemical technology that operated through saponification and takes the form of programmable lifelike droplets. It serves both as a model system for morphological computing that enables us to observe emergent events and also as an experimental technology that can be shaped to carry out specific design programs. Unlike mechanical systems, which operate at equilibrium conditions and require external energy to be applied to perform useful work, morphological computing's outputs are consistent with the performance of non-equilibrium systems, which may be described by the laws of complexity. Unlike the inert objects that machines are built from, the "actants" that comprise assemblages possess their own agency and result in a range of phenomena such as, emergence. Therefore, technological opportunities exist to directly and dynamically manipulate materials at far from equilibrium states in complex ways by developing spatial chemical programs. Yet, as a design platform, morphological computing is not a discrete set of components but consists of an entangled system of agents that multi task interchangeably as fabric, software and hardware. For example, the Bütschli system exemplifies how tightly fabric, software and hardware are coupled in morphological computing processes. Bütschli droplets are produced when oil molecules, which may be thought of as a dynamic chemical program (both fabric and software), encounter alkali (both fabric and software) to produce soap-like crystals (both fabric and hardware), which form microstructures. The overall performance of this system is shaped by many environmental conditions such as, temperature and local conditions such as, the movement of the droplets and the speed at which crystallization occurs [28]. Bütschli droplets demonstrate how different morphological computing outputs are to those of machines [29]. The hardware principles of this system are embodied in the droplet behaviour, which are not manufactured but self-assembled. When the alkaline solution is added to an oil field to produce the Bütschli system, individual dynamic droplets 157 Morphological Computing and Design spontaneously arise from a field of chemical activity, which spreads out and breaks up into agents that are about a millimetre in diameter. The dynamic droplets of the Bütschli system are therefore not objects but agents since they possess an internal force that is powered by chemistry (or metabolism) and do not need an external energy source for them to exert their effects. Some of the properties are lifelike. For example, dynamic droplets can move around their environment, sense it and make products as a side effect of their metabolism. They spontaneously form loose, reversible interactions as parallel processors that are not organized hierarchically and form the basis of their "assemblage" operating system. This operational "looseness" in their order confers the assemblage with its robustness, flexibility and capacity to deal with external events. Perhaps surprisingly, the outputs of morphological computing systems are relatively conservative and predictable within limits, except when a system reaches tipping points, where dramatic changes in outputs can be observed such as, simultaneous shape and behaviour in Bütschli droplets, a process that is still not fully understood Figure 3. Figure 3: Two initially distinct assemblages of Bütschli droplets are produced simultaneously under identical conditions. A smaller population moves towards a larger one and the two assemblage formations entangle with each other. In a complex series of interactions, a tipping point of organization is reached in the composited assemblages where the individual droplets that made up both the original formations, suddenly individuate and synchronously change their morphology and behaviour. The technological principles of morphological computing may be demonstrated in Bütschli droplets by changing the internal and external chemistry of the system. For example, by adding a mineral solution such as, 1M copper II sulphate to the alkaline droplets of the Bütschli system (3M sodium hydroxide), insoluble crystals (Copper II carbonate) are produced when the droplet body also comes in contact with dissolved carbon dioxide, see Figure 4. Bütschli droplets will also respond to changes in the surface tension and viscosity of their medium and when pure ethanol is added to an olive oil field containing Bütschli droplets, they will move toward it by chemotaxis Figure 5. Yet, systems that can be easily operated at the human scale are needed if morphological computing techniques are to be used widely by designers. While unmodified Bütschli droplets are around a milimeter in diameter, they can be designed to reach several centimetres in diameter by slowing down their metabolism by adding a small amount of 158 Morphological Computing and Design Figure 4: Microstructures within the body of a modified Bütschli droplet produced on contact with a mineral solution and dissolved carbon dioxide. inhibitor. Morphological computing systems are susceptible to many influences and because they possesses innate agency, the platform itself has the capacity to co-design events, which may be directed by humans through morphological computing techniques. For example, when Bütschli droplets are placed within a constrained space, the chemical patterns that govern their interactions are revealed. An installation was designed for the Synth-Ethic group show at the Bio fiction festival that was held at the Natural History Museum in Vienna, in April 2011. Modified Bütschli droplets were introduced into a constrained space of 2cms diameter, which may be provoked by the diffusion and reaction patterns of chemical systems that Alan Turing proposed governed biological processes, such as gastrulation and animal skin patterns, specifically "dappling" [30]. Today, these effects are attributed to the actions of information molecules such as, RNA. However, since the lifelike properties of the Bütschli system are chemical, Turing's theory is actually responsible for the sinusoidal patterns produced in the system Figure 6. A variety of chemical assemblages can operate fully as forms of morphological computation in a range of specific contexts. For example, in the Hylozoic Ground installation designed by architect Philip Beesley for the Venice 2010 Architecture Biennale, I designed a range of chemical "organs" for this cybernetic system, with different kinds of metabolisms. Liesegang ring plates are a special preparation of Liesegang rings [31] that were made especially for the Hylozoic Ground installation. Rather than being situated within a laboratory test tube, they are constrained within a digitally fabricated contained with a narrow Perspex space. Light can pass through the activated gel matrix to reveal the fine details of the diffusionprecipitation interactions of mineral solutions, which from a design perspective, serve to chemically mark the passage of time. Other techniques used hygroscopic materials to 159 Morphological Computing and Design Figure 5: Bütschli droplets have accumulated around a source of ethanol through chemotaxis, which is likely to be the consequence of rapid surface tension instabilities following diffusion of the organic substance through the oil medium. Bütschli droplets that have been attracted into a field by ethanol rapidly become still as surface tension instabilities settle. harvested water from the atmosphere and swell, drawing the beginnings of a fluid matrix towards the installation as a gesture towards increasing the context for lifelike events to take place within the cybernetic system. Modified Bütschli droplets were also incorporated into this installation that were chemically "programmed" with brightly coloured salts, which changed colour in the presence of carbon dioxide. They were poetically considered as an artificial smell and taste system, whose performance was entangled with the presence of humans that breathed out activating respiratory gases Figure 7. While machine outcomes are highly predictable, assemblages operate within a range of operational limits that are typical of probabilistic systems, which are defined by internal and external conditions. The complex performance of assemblages creates a design and engineering platform that has the potential to evade the traditional binary divisions between various systems and modalities such as, Nature/machine, humanism/environmentalism and matter/information. In dissolving these divisions morphological computing ultimately increases the connectivity of matter with the environment and provides an operating system of assemblages that enables us to design ecologies. Morphological computing may therefore be imagined through the language of ecology and its outcomes and may be orchestrated using "soft" control techniques, which coerce and shape outcomes, rather than dictate them. From a design perspective, morphological computing may even be scaled to urban and ecological dimensions in the speculative, and real project, Future Venice [32]. This proposes to sustainably reclaim the city by growing an artificial limestone reef underneath 160 Morphological Computing and Design Figure 6: Modified Bütschli droplets produce Turing bands at the Synth-ethic group show, Bio Fiction festival, Natural History Museum of Vienna, 2011. to spread the point load and stop it from sinking into the soft delta soils on which it is founded. The sensory and motor systems of this computer are coupled in programmable droplets that are able to move around in the waterways. The activities of the droplets can be directed through their metabolisms, which are engineered to move away from the light and produce a limestone-like substance, or 'biocrete' when they are at rest. In the light-soaked waterways the droplets move towards the darkened foundations of the city and accrete around the woodpiles on which the city stands, and use dissolved minerals and carbon dioxide to accrete an artificial limestone like structure under the foundations of the city and steadies it on the underlying soft delta soils Figure 8. A natural version of this process can be observed around the lagoon side and the canals, which is being orchestrated by the natural marine life. It is proposed that the natural computer could work alongside the organisms to co-construct an architecture that is mutually beneficial to the marine ecology and the city. Importantly, should the environmental conditions change and the lagoon dries out – say for example, Pietro Tiatini and his colleagues succeed in anthropogenically lifting the city by pumping seawater into its deflated aquifers [33], or if when the MOSES gates are raised in 2014 the native ecology reaches a catastrophic tipping point then the natural computer can re-appropriate its actions so that as the waters subside, instead of growing a reef, the droplets coat the woodpiles with a protective layer of "biocrete" that stops them rotting when they are exposed to the air. Indeed, morphological computing processes could theoretically be applied to the whole bioregion of Venice as a form of environmental design. Developing the right kinds of metabolisms and spatial programs could give rise to tactics that generate, new relationships between natural and artificial actants, and become the bedrock for forging life-promoting, synthetic ecologies. Morphological computing may then become a design 161 Morphological Computing and Design Figure 7: A range of morphological computing systems were embedded in the Hylozoic Ground installation by Philip Beesley, that were installed as dynamic chemical "organs," which responded to environmental cues and were exhibited at the 2010 Venice Architecture Biennale. Modified Bütschli droplets can be seen centre image as yellow flask in the cybernetic field and hygroscopic materials appear as a field of suspended orange structures to the right of the photograph. Photograph courtesy PBAI, 2010. practice of shaping overlapping spatial programs and developing tactics that enable a constant flux between fabric, space, structure and location, where the outputs of the system do not imitate Nature but operate according to "low level" programming principles. Yet, morphological computing does not propose a comprehensive solution to Venice's precarious future – or indeed our legacy our environmental woes. It is simply a method of observing, engaging and designing with dynamic material systems, which are limited by the performativity of matter, environmental context and the realm of influence that human design can have over any realm. A design practice based on morphological computing techniques does not attempt to "solve" the inevitable changes that accompany a dynamic and lively environment. Instead, it proposes a convergent platform that enriches the available opportunities by which human and nonhuman communities may respond together to environmental events and challenges as co-designers of shared futures. Working with assemblages is challenging, as they require a new conceptual and practical toolset so that it may be possible to identify appropriate challenges for their application and imagine, design and engineer with them in ways that best work with their innate characteristics. They are very different to object-cantered industrial methods as they are sensitive to and forge connections with the environment. Yet, this provides a different kind of production platform where a synthetic relationship between technology and our surroundings may be developed to shape networks of sustained interactions to achieve outputs, like growth and repair. Morphological computing therefore not only exists as a series of exemplary, prototype projects but also operates as a design platform that strives to discover something new about reality. By working with matter at far from equilibrium states, it is possible to challenge our expectations of the material realm and couple systems together to produce surprising effects using constructivist approaches that engage its physicality in the performance of a system. Yet, it is not essential to know everything about nonlinear materials to design effectively with them, as the outputs are emergent and surprising, so 162 Morphological Computing and Design Figure 8: Natural mineral formations around the Venice waterways produced by morphological computing processes. the design process is a journey of discovery where designers of morphological computing practices are also the co-authors of the emergent processes of the systems. The fundamental design units of morphological computing are not "objects" but leaky systems that may be formed directly through loose chemical and biological assemblages. They facilitate enable information flow (droplets, proteins, DNA), and ideally, can remain open (e.g. cells) throughout the computational process. Yet, design with morphological computing is not exclusively concerned with "empirical" outcomes but also engages "entangled knots of mutual feeling and action." [34]. Morphological computing therefore has the potential to entangle aesthetics with realm of computing, as proposed by Gregory Bateson [35] that also correlate with new insights into the behaviour of matter. Morphological computing does not use top-down instructive programming such as, the modification of DNA that lies behind the technological intentions of synthetic biology but engages with an orchestration of matter, through overlapping complex units which result in a new engagement with materiality, which has an elevated status as it is participatory in the experiment, not passive [36]. Indeed, it helps designers reveal the strangeness and dynamic potential of a material world that we thought we already knew. Notably, morphological computing strives to keep its computational outputs "open" and therefore offers an adaptable, perhaps even evolvable production platform. It is therefore never fully complete but flexible and modifiable throughout its lifespan, although these characteristics may differ by degree as the entropy of the system inevitably changes. Its technological platform embraces Erwin Schrödinger's notion of "evading" decay towards equilibrium [37] by using the spatial properties of matter to "insert time and space" into the system. From a design perspective, the outputs of morphological computing may be thought of as exhibiting some of the properties of living systems 163 Morphological Computing and Design although they may not possess the status of being fully alive. Ultimately morphological computing may prove an essential platform in developing ecologically compatible manufacturing practices, or in synthesizing new kinds of ecological systems. To keep its computational processes "open" morphological computing requires a unique kind of infrastructure to traditional technologies that are unique to the particular system under study. These infrastructures may be thought of as being "elemental" in character and require the continual flow of a medium as well as the provision of energy to support the computation process such as, air or water. They may provide a range of functions that are usually associated with the basic needs of living things such as, providing sustenance, removing waste products and delaying the onset of thermodynamic equilibrium as much as possible and prolonging its lifelike effects for as long as possible. Currently, morphological computing proposes a new manufacturing platform with the capacity to become independent of – but not unresponsive to humans or other living systems. The solution provided is an engaged collaboration and co-authorship between a population of design agents and human designers, or "programmers." The outputs of morphological computing may act independently and produce effects on a site or environment to modify it, act as a carrier system to move matter through time and space, or integrate with other agents and assemblages to produce highly contingent, synthetic outputs. Although morphological computing proposes a terrain in which "life" may be an event, it does not aim to be a form of artificial life per se. Yet, the emergence of fully "alive" systems from a morphological computing platform is a possibility, since "autocatalytic" sets of interacting agents [38] could conceivably become fully autopoietic with a unique life span and even a life cycle. Stephen Jay Gould noted that there are many types of evolution [39] and morphological computing could be considered as a way of conducting experiments in re-playing "the tape of life" [40]. Of course, morphological computing would not be weaving its tapestry from life's origins, but from some consensual point in an "unevenly distributed present." Potentially then, this new technological platform sets out the fertile conditions that may provoke various kinds of evolution, which may ultimately result in locally produced, artificial biological systems, or "natures" – as entanglements of technology, ecology and culture [41]. Yet despite the relentless variation in the system there are shared patterns, regularities and principles with have definable limits, which do not restrict the production of novelty in the system, but provoke it. In the far future, morphological computing may help us build artificial ecosystems. Project Persephone, which is one of the Icarus Interstellar group of initiatives that aim to catalyse the construction of a worldship within a hundred years, creates a context for exploring ecological questions. Yet, these are not framed within the current industrial way of thinking where something that is "good" for the environment consumes it less, or more considerately but instead regards its design and engineering goals as promoting lifelike events. Persephone therefore seeks to understand ecopoiesis [42], which is a term applied to the first stages of terraforming a lifeless planet. Ecopoiesis is different to terraforming in that it works to set up any kind of dynamic material system, which may be thought of as a primitive ecology, so that it may one day be possible to boot start a planet from one that is dead and lifeless to being able to establish processes that 164 Morphological Computing and Design enable the spontaneous exchange and transformation of molecules. These processes may be facilitated using morphological computing techniques to establish a series of organizing hubs that begin to terraform the system by transforming the primitive ecosystem into one that is more Earth-like Figure 9. However, the process of ecopoiesis is not Figure 9: Project Persephone proposes to develop her life-promoting soils as a form of morphological computing to discover new ways of supporting human settlement beyond Earth's natural environment. Drawing by Phil Watson & Jonathan Morris, 2013. just a chemical equation but requires spatial considerations that invoke morphological computing techniques by separating chemical reactions in time and space. Biological systems are experts at this process, since eukaryotic cells possesses an organelle composed of an ordered system of cavernae called the "endoplasmic reticulum." This is made up of labyrinthine structures that process specific molecules inside eukaryotic cells by transporting them from one caverna to the other, in a stack of cavernae. Accordingly, Persephone also resists entropic decay by using soils as the technology that can do this. In some ways soils may be likened to a giant, series of cavernae that although are much more disordered than the endoplasmic reticulum in eukaryotes, nonetheless provide a system that spatially orders chemistry and catalyses interactions between molecules. Indeed, soils and are an amazing, yet largely overlooked natural technology, which are the basis for all life on earth. They have also formed the fertile conditions and material foundations on which our greatest cities have been built. Yet, while Earth's planetary systems are open, since they continually receive matter from space and energy from light, Persephone has to work harder at maintaining a materially rich system by feeding on space junk, asteroids or electromagnetic radiation from stars that can be converted into soil-making substances. Although Earth's soils have spontaneously evolved over many tens of thousands of years through a combination of physical, chemical and biologically processes, Persephone cannot wait that long and therefore uses advanced technologies to generate her life-promoting soil infrastructures. Artificial soil prototypes are being 165 Morphological Computing and Design designed using activated gels and inorganic chemistries and are hoped to provide insights into how we may promote our soils on Earth, as well as support human existence beyond our pale blue planet Figure 10. Figure 10: Detail of Persephone's synthetic soils. Drawing by Phil Watson & Jonathan Morris, 2013. What Are the Main Challenges in Order to Turn Morphological Computation from an Idea into a Field of Which Institutes Will Be Named After Morphological computing needs to be expressed in relation to the world's greatest challenges so that general audiences can clearly understand the context for their application and how they work. Lessons must be learned from the breakthrough technologies of genetic modification and cloning where ethical and moral questions continue to be raised about the uptake of these technologies, despite their many benefits. Yet, it is equally as important not just to focus the possible negative impacts of the new technology but also to provide clear examples of how morphological computing enables us to innovate and address challenging issues in new ways. The story of morphological computing therefore needs a clear narrative about its history, purpose and relationship with our environment. Yet, this cannot be achieved by one discipline alone and therefore it is essential for multi-disciplinary collaborations between technologists and organizations that can deal with the ethical, moral and environmental aspects of this emerging practice, so that critical issues can be sensibly discussed in open public forums. Effective public engagement is needed that is not about instructing non-scientists about the technical details of a particular field of research but in finding 166 Morphological Computing and Design ways that can creatively position the value of scientific developments so that it is clear where the research is heading and inspires wider interest. A broad range of international practitioners from different disciplines that are not familiar with classical computing techniques and concepts should also be associated with the technological development of new computing platforms. These may include designers, architects, ecologists, science fiction authors and artists that may imagine a diverse set of applications for morphological computing and assist the realization of these ideas using visualizations to provoke further discussion about their relevance to the greatest challenges that we currently face. It is also necessary to consider inclusivity and how access to the technology may be democratized by thinking about its relevance to distributed arts and domestic practices such as, BioArt or 3D printing, so that non-specialists can see how they may use the technology themselves and what the benefits will be on a very personal level. Ideally, morphological computing needs to be imagined in contexts that are as accessible as personal computing and not exclusive to institutions, industry or the developed world, where only a few people benefit from investment in this field of research. How can Morphological Computation Facilitate the Interaction of Humans and Machines, both in Handling and Programming? In assessing the success of morphological computing techniques it is necessary to develop evaluation methods that exceed the performance criteria usually applied to machines, such as power and efficiency, which do not adequately appreciate the strengths of its unique lifelike operations such as, robustness, resilience, adaptability, responsiveness and unpredictability. Indeed, morphological computing is a fundamentally creative practice that offers a range of novel outputs by coupling different kinds of agents together to provide new tools, but may also precipitate encounters of beauty and awe through a conundrum of dynamic, emergent phenomena, some of which resist classification and require further study, or analysis through the lens of process philosophy. It confronts a range of classical assumptions about computing as it is a collaborative enterprise that requires us consider how "agency" is distributed through its networks and offers a degree of soft control, rather than absolute instruction. Skilled designers and programmers may therefore develop morphological computing systems in ways that differentially distribute of agency across a network so that designers and engineers can meaningfully shape the outcomes. Morphological computing may ultimately enable us to access the technological qualities of our lively planet so that we may address some the significant ecological challenges that we face on this incredible planet in new ways and become better equipped to deal with the dynamic reality of our restless Earth. 167 Morphological Computing and Design Bibliography [1] Prigogine, I. "The End of Certainty, Time, Chaos and the New Laws of Nature", The Free Press, New York, NY, pp. 67, 1996. [2] Pfeifer R. and Iida F. 2005. Morphological computation: Connecting body, brain and environment. Japanese Scientific Monthly, 58(2), pp. 48-54. [3] Hauser, H. 2013. Morphological computation and soft robotics, a perfect match? Soft robotics and morphological computing, 2013 International Workshop, Ascona, Switzerland 14-19 July, 2013. Workshop Digest. p. K-12. [4] Cagdas D. Onal, Xin Chen, George M. Whitesides, and Daniela Rus, Soft mobile robots with on-board chemical pressure generation. Onal, C.D. Chen, X. Whitesides, G.M. and Rus, D. 2011. Soft Mobile Robots with On-Board Chemical Pressure Generation, International Symposium on Robotics Research (ISRR) , Flagstaff, AZ. pp. 1-16. [5] Hanczyc, M.M. Toyota, T. Ikegami, T. Packard, N.H. and Sugawara, T. 2007. Fatty acid chemistry at the oil-water interface: Self-propelled oil droplets. Journal of the American Chemical Society, 129(30), pp. 9386 9391. [6] Toyota, T, Maru, N, Hanczyc, M.M. Ikegami, T. Sugawara, T. 2009. Self-Propelled Oil Droplets Consuming "Fuel" Surfactant. Journal of the American Chemical Society, 131(14), pp. 5012-5013. [7] Snow, C.P. 1959. The Two Cultures. London: Cambridge University Press. [8] Denning P. J. (2007) Computing is a natural science, Commun. ACM, 50(7), pp. 13-18. [9] MacLennan, B. 2011. Artificial morphogenesis as an example of embodied computation. International Journal of Unconventional Computing, 7(12), pp. 3-23. [10] Paun G. 2005. Bio-inspired Computing Paradigms (Natural Computing) in Unconventional Programming Paradigms. Lecture Notes in Computer Science, 3566, pp. 155-160. [11] Oltean, M. 2009. Unconventional Computing: a short introduction. Studia Universitatis Babes-Bolyai Series Informatica, 54(1), pp. 85-96. [12] Adamatzky, A. Bull, L. De Lacy Costello, B. Stepney, S. and Teuscher, C. eds. 2007. Unconventional computing. Beckington, UK: Luniver Press. [13] Armstrong, R. 2011. Unconventional computing in the built environment, International Journal of Nanotechnology and Molecular Computation, 3(1), pp. 1-12. [14] Adamatzky, A. Armstrong, R. Jones, J. and Gunji, Y.K. 2013. On creativity of slime mould, International Journal of General Systems, 42(5), pp. 441-457. 168 Morphological Computing and Design [15] Adamatzky A, De Lacy Costello B, Ratcliffe NM. 2002. Experimental reactiondiffusion pre-processor for shape recognition. Physics Letters A, 297, pp. 344-352. [16] Adamatzky A, De Lacy Costello B. 2003. Reaction-diffusion path planning in a hybrid chemical and cellular-automaton processors. Chaos, Solitons and Fractals, 16, pp. 727-736. [17] Seibt, J. 15 October 2012. Process Philosophy, Stanford Encyclopedia of Philosophy. [online] Available at: http://plato.stanford.edu/entries/process-philosophy/. [Accessed 16 April 2013]. [18] Armstrong, R. 11 July 2013. Rachel Armstrong on biomimicry as parametric snake oil. The Architectural Review. [online] Available at: http://www.architecturalreview.com/reviews/ biomimicry-parametric-snake-oil/8650000.article [Accessed 18 July 2013]. [19] Myers, W. and Antonelli, P. 2013. Bio Design: Nature, Science, Creativity. London: Thames & Hudson /New York: MOMA, p.10-17. [20] Larssen, M. 2012. Dune. In: W. Myers and Antonelli, eds. Bio Design: Nature Science Creativity, MOMA/Thames and Hudson, 2012. pp. 62-65. [21] Mama, J. and van Heerdan, C. 2012. Microbial home. In: W. Myers and Antonelli, eds. Bio Design: Nature Science Creativity, MOMA/Thames and Hudson, 2012. pp. 96-101. [22] Whitelaw, M. 2003. Morphogenetics: generative processes in the work of Driessens and Verstappen. Digital Creativity, 14(1), pp. 43-53. [23] Deleuze, G. and Guattari, F. 1979. Thousand Plateaus: Capitalism and schizophrenia (Athlone Contemporary European Thinkers). London: The Athlone Press. [24] Latour, B. 2005. Reassembling the Social: An Introduction to Actor-NetworkTheory. Oxford: Oxford University Press, p. 54. [25] Bennett, J. 2010. Vibrant matter: A political ecology of things. Durham: Duke University Press, p. 23. [26] Deleuze, G. and Guattari, F. 1986. Kafka: Toward a Minor Literature. Minneapolis: University of Minnesota Press, p. 22. [27] Bütschli, O. 1892. Untersuchungen ueber microscopische Schaume und das Protoplasma, Leipzig. [28] Armstrong, R. and Hanczyc, M.M. 2013. Bütschli dynamic droplet system. Artificial Life Journal, 19(3-4), pp. 331-346. [29] Armstrong, R. 2013. Alternative biologies. In: M. More and N.V. More, eds. The transhumanist reader: Classical and contemporary essays on the science, technology, philosophy of the human future. Chichester: John Wiley and Sons, pp.100-109. 169 Morphological Computing and Design [30] Turing, A.M. 1952. The Chemical Basis of Morphogenesis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 237(641), pp. 37-72. [31] Liesegang, R. E. 1869. Ueber einige Eigenschaften von Gallerten, Naturwissenschaftliche Wochenschrift, 11(30), pp. 353-362. [32] Armstrong, R. 2012. Future Venice. In Myers, W. and Antonelli, P. eds. Bio Design: Nature, Science, Creativity. London: Thames & Hudson /New York: MOMA, pp.7273. [33] Castelletto, N. Ferronato, M. Gambolati, G. Putti, M. and Teatini, P. 2008. Can Venice be raised by pumping water underground? A pilot project to help decide, Water Resources Research, 44, W01408, doi:10.1029/2007WR006177. [34] Spuybroek, L (2011) The sympathy of things: Ruskin and the ecology of design, NAi010 publishers in association with V2, Rotterdam, Netherlands, pp.322. [35] Bateson, G.(1979) Mind and nature: A necessary unity. Cresskill, NJ: Hampton Press, pp. 12. [36] Bennett, J. (2010) Vibrant Matter: A Political Ecology of Things. (2009). Duke University Press, Durham and London, pp. xvi. [37] Schrödinger, E. (1944) What is life? Based on lectures delivered under the auspices of the Dublin Institute for Advanced Studies at Trinity College, Dublin, in February 1943. Available at: http://whatislife.stanford.edu/LoCo_files/What-is-Life.pdf Accessed October 2012. [38] Kauffman, S. (2008) Reinventing the Sacred: A New View of Science, Reason, and Religion. Basic Books, NY, pp.55. [39] Gould, S.J. (1994) The evolution of life on the earth. Scientific American, 271, no 4 (October 1994):84. [40] Simon Conway Morris and Stephen Jay Gould, "Showdown on the Burgess Shale," Natural History magazine, 107 (10): 48-55. [41] Van Mensvoort, K. and Grievink, H.J. 2012. Next Nature: Nature Changes Along with Us. Barcelona: Actar. [42] Mautner, M. N. 2000. Seeding the universe with life. Washington D.C.: Legacy Books. 170 Abstract: This short essay introduces and states my opinion on future perspectives of morphological computation from the perspective of molecular computing. An analysis of morphological computation is introduced mainly from a viewpoint of "scales" and "interactions." Some examples of biomolecular computing, especially DNA computing, are briefly described in terms of morphological computation. Moreover, possible future connection and possibilities between morphological computation and a new field called "molecular robotics" are mentioned. Morphological Computation at the Molecular Scale Introduction 1 The word "computation" is now a well-understood idea among people, thanks to the invention and rapid progress of the modern silicon-based computer hardware and software. Now the computers are found everywhere; people cannot live their life without them. Because of this reason, we are unconsciously biased and tend to forget the other options of computation. Of course silicon-based computers are not the only way to execute computation; however, because computers with current architectures are almost incomparable to the other types of computation in terms of human interest like spatiotemporal scales and costs, we didn't give serious attention to other possibilities until the recent emergence of new frontiers that conventional computing may have difficulties to adopt. In the next section, I will introduce an analysis of morphological computation, and the following sections will explain my opinion why this idea may become important and has a huge potential in the future, especially in terms of molecular computing and molecular robotics. Morphological Computation: A Brief Comparison From one of the broadest and widely known definitions, computation can be described as "a goal-oriented process that describes and transforms information [4, 2]." The point here is that we need a certain medium to execute this process. Conventional computation uses digital circuits and electrons; on the other hand, morphological computation uses "things" that can interact with environment. The largest difference here is the presence of the substance with certain forms that represents its states. In short, we cannot distinguish the states of electric circuits only by the appearance of the chips; on the other hand, we can distinguish or estimate morphological computers' states because "forms" themselves are related to the states. Therefore, morphological computation can be regarded as a complementary concept compared to the conventional (silicon-based) computation, in terms of physical substance with interaction between environment and states. This relation between environment and the substance also suggests that the calculation process is taken over at certain extent by the environment, because physical laws of the environment can implicitly determine forces that can deform or move the objects. These points give some fundamental characteristics, for instance: 1. Complexity. Conventional computation usually excels in complexity thanks to the successful framework of hardware and software, which relies on hierarchy and modularity of the system; on the other hand, morphological computation needs to avoid unintended crosstalks in order to increase complexity, because communication between objects are based on interactions all simultaneously happening via the environment. Morphological computation needs to utilize the characteristics of the environment itself (i.e. uses the physical rules as a part of the rules/information for computing) in order to decrease this error. 1This essay was written in Oct. 2012. Several additional references and footnotes were added on 07/28/2014. 172 Morphological Computation at the Molecular Scale 2. Calculation speed. Silicon-based computation is based on a series of electric circuits, and each "reaction" is fast; on the other hand, morphological computation is often realized in parallel reactions, although each reaction is relatively slow, mainly because the reaction is mediated by the presence of substance and physical rules of the environment. Only if the objective we want to solve is well suited to the characteristics of the environment, then morphological computation may become valuable. 3. Abstraction. As a compensation of realizing versatility in calculation, computation often requires a certain conversion process between human inputs/outputs and the medium (often those are represented as digital information). This abstraction process cannot be avoided, and especially in case of conventional computation, they often consume a lot of energy, and also the process itself become complicated. In contrast, morphological computation usually doesn't explicitly require such an abstraction process: whether we determine 0s and 1s, shapes and forms are predetermined (or abstracted beforehand) and given directly in terms of objects, so the conversion process is not needed. In case of morphological computation, input starts from objects with certain form, calculations directly use those objects, and then the output would also be the objects in different form that also can be used directly. Especially if those inputs/outputs don't require further human interaction, additional conversion process will not be needed. These points suggest that most of the benefits of morphological computation will likely be hindered in the usual usage of computation, especially in our human life scale such as length and time.2 Although morphological computation still has quite large advantages in energy consumption, cost and in some particular cases (like in case of physical object calculation), conventional computation still exceeds (or at least we doesn't feel them as a problem) in general. Scales that Things Matter In the previous section, a comparison between morphological and conventional computation has been made and revealed some differences in characteristics. The advantage of morphological computation is effective in certain cases, although we cannot (or very difficult to) utilize them in our life scale, mainly because of the properties of the environment. However, the advantage of morphological computation will come under the spotlight if we change the environment into smaller scales, especially to the scale that governed by Brownian motions. Biomolecular systems can be considered as the most obvious examples that effectively utilize the characteristics of morphological computation. Form or shape of the molecule itself and the information it carries are two sides of the same coin in this scale. For example, DNA uses sequence in order to store information, and those sequences are 2Update (07/27/2014): some discussions on scalability of morphological computing were given in ref. [5]. 173 Morphological Computation at the Molecular Scale represented by an array of four types of bases: hybridization utilizes the specificity of those sequences based on Watson–Crick base pair complementarity (interactions between molecules) and Brownian motion of the molecules (environmental rules). Protein folding uses the sequence of amino acids for primary information, but also utilizes the Brownian motion and other chaperone proteins in the environment to bring molecules into the exact shapes. These examples clearly show that those effective usages of environmental rules and shapes or forms of molecules can be considered as a certain kind of morphological computation and play important roles in one of the most complex systems - life. Although the mechanisms we use in those systems are completely different than that of conventional computers, it is obvious that we can still make a sophisticated system based on this strategy of utilizing environmental rules and shapes at the molecular scale. A great hint of how to design morphological computation from molecules is shown in those examples. DNA Computing and Morphological Computation DNA computing is one of the fields that learned from nature how to utilize the characteristics of the environment and forms of molecules at the nanoscale and using them in terms of computation. Sequence information and hybridization is used for the calculation from the beginning of this area, started by Adelman [3]. Recent developments of structural DNA nanotechnology, founded by Nadrian Seeman, pushed the frontline of this computation forward by introducing the "shapes" of the molecules, which expands not only in 1-D (double helix), but also into 2-D and 3-D DNA nanostructures [7]. From the aspect of morphological computation, for example, we can consider that DNA tiles [8] are implicitly using certain restriction by the shapes of the motifs, not only the sequence information, in order to form desirable 2-D nanostructures with patterns. Each tile has several single-stranded ends, called "sticky ends;" if they meet each other and have matching, i.e. complementary sequences, then these strands are able to connect with each other and form a double helix. Of course this description only writes down the rules of the connection. In the real environment, those tiles are moving inside the solution by Brownian motion. If sticky ends randomly meet each other in this environment, then those rules are effective and the connection described above will happen. The shape of the molecule defines and restricts the assembly only to the desirable formation, and the environment enables that formation process: the shapes of those molecules themselves can be considered as (being a part of) the "rules" of this system. Moreover, we can use not only the complementarity of DNA strands but also the stacking effect and shapes of the structure as a rule of this type of computation. Woo and Rothemund [9] made DNA Origamis with patches of stacking ends. A site with stacking ends can be considered as "1", and without stacking ends as "0". Two ends of 2-D rectangular origami are coded with those binary sequences, and only the matchedends can connect each other. This rule does not use the sequence specificity of DNA, but instead uses the "shape and binary order" of the molecule and the non-specific interactions of DNA in order to create specificity in the ends and thus certain computation 174 Morphological Computation at the Molecular Scale is realized. Those combinations of specific interactions and nonspecific interactions plus morphology will be a key to design sophisticated systems by molecules in future. Future Molecular Robotics and Morphological Computation Current trends in "engineering" biomolecular system from bottom-up lead us to the idea of making a certain autonomous system with sensors, processors and actuators, all by molecules. We call this idea "molecular robotics" [6]; researchers in Japan from wide varieties of background organized a research group [1] for this new emerging research field and projects has already been started towards this direction.3 Computations in the future molecular robots will be executed inside the robot itself; also several types of molecules will likely to be an input/output of the computation, because those will be transferred from/to other subsystems which are also made by molecules, such as sensors and actuators. DNA will play one of the pivotal roles in this calculation process of molecular robots; knowledge and know-hows of DNA computing will also be used in this area. These expectations suggest that the idea of morphological computation will also play an important role in the field of molecular robotics. However, future molecular robotics may not use only one type of molecules: the conversion process between the molecules for computation (such as DNA) and other types of molecules will be one of the points that we need to solve. The idea of morphological computation may provide a hint towards such a solution. Learning from nature, for example, the central dogma of molecular biology can be interpreted as a kind of hierarchy segmented by several types of different molecules with information (DNA, RNA and Proteins), and specially designed molecules (DNA Polymerase, RNA Polymerase and Ribosome) that can coordinate two different types of molecules, and thus can convert and transfer information between those layers. Those molecules use weak interactions and also the shape of the molecule itself to connect the molecules in different layers: interactions and shapes, those two can be considered as a common language among various types of molecules, when properly designed. If we can mimic this idea and engineer those properties for the future molecular robotics, the problem of conversion and also the crosstalk will be solved. Conclusion This essay overviewed the strength of morphological computation mainly from a perspective of scales and showed that the molecular scale would be one of the environments at which morphological computation can utilize its potentials. Some examples from molecular computing, especially DNA computing and structural DNA nanotechnology, are introduced in terms of morphological computation, and it is pointed out that the environment, specific interactions, non-specific interactions and the shape of molecules are playing the key role at this scale. Moreover, the idea of molecular robotics is introduced; 3Update (07/27/2014): our project, the development of molecular robots equipped with sensors and intelligence, is currently funded by the Ministry of Education, Culture, Sports, Science, and Technology, Japan (MEXT) as a 5-year project. 175 Morphological Computation at the Molecular Scale morphological computation may provide an insight to solve future problems in this area, such as conversions between different types of molecules and crosstalk. Although many points are still vague and not mentioned in a quantitative way, the importance of morphological computation in the molecular scale is obvious. I believe further investigations on molecular computing and also molecular robotics will provide actual examples, and those examples will also give us an understanding towards unified view of morphological computation across scales in future. Bibliography [1] Molecular robotics research group, 2010. URL http://www.molbot.org. [2] The joint task force for computing curricula 2005, Sep. 2005. URL http://www.acm. org/education/curric_vols/CC2005-March06Final.pdf. [3] L. Adleman. Molecular computation of solutions to combinatorial problems. Science (New York, NY), 266(5187):1021–1024, November 1994. [4] P. J. Denning, D. E. Comer, D. Gries, M. C. Mulder, A. Tucker, A. J. Turner, and P. R. Young. Computing as a discipline. Communications of the ACM, 32(1):9–23, February 1989. [5] R. M. Füchslin, A. Dzyakanchuk, D. Flumini, H. Hauser, K. J. Hunt, R. H. Luchsinger, B. Reller, S. Scheidegger, and R. Walker. Morphological computation and morphological control: Steps toward a formal theory and applications. Artificial Life, 19(1):9–34, January 2013. [6] S. Murata, A. Konagaya, S. Kobayashi, H. Saito, and H. M. Molecular Robotics: A New Paradigm for Artifacts. New Generation Computing, 31(1):27–45, January 2013. [7] N. C. Seeman. Nanomaterials Based on DNA. Annual Review of Biochemistry, 79: 65–87, July 2010. [8] E. Winfree, F. Liu, L. A. Wenzler, and N. C. Seeman. Design and self-assembly of two-dimensional DNA crystals. Nature, 394(6693):539–544, August 1998. [9] S. Woo and P. W. K. Rothemund. Programmable molecular recognition based on the geometry of DNA nanostructures. Nature Chemistry, 3:620–627, July 2011. 176 Abstract: In this contribution a broad perspective on morphological computation is introduced. We shall argue that shape is a special feature of spatial relations and shall advocate a computation model that takes into account spatial relationships between components of the computation. Morphological Computation – A Broad Perspective Current Definitions When trying to get a clear picture of a certain term, it is often useful to look at the opposite to see whether that term carries any meaning. What would be the opposite to morphological computation? One guess would be "morphological non-computation," and this term might indeed have some legitimacy, but not in our context. Rather "nonmorphological computation" would be the proper choice of words for the purpose of this discussion. Some would say, "amorphous computation," a term that has indeed been used in the recent past [1]. The authors write: "An amorphous computing medium is a system of irregularly placed, asynchronous, locally interacting computing elements. We can model this medium as a collection of "computational particles" sprinkled irregularly on a surface or mixed throughout a volume. [. . . ] Each particle has modest computing power and a modest amount of memory. The particles are not synchronized, although we assume they compute at similar speeds, since they are all fabricated by the same process. The particles are all programmed identically, although each one has means for storing local state and for generating random numbers. In general, the particles do not have any a priori knowledge of their positions or orientations."[1] In other words, amorphous computing is massively parallel, assumes identical elements (at least ideally) of limited capability and homogeneity of substrate, no a priori knowledge of location of elements relative to each other, but emergence of behaviour from local interactions. Local interactions are perhaps best described as a requirement of the massive parallelity in these systems. The identity of elements is a vestige of engineering thinking, which in the real world can only be an approximation. Overall, the aim of amorphous computation is algorithmic abstraction. Let us now turn to a definition of morphological computing / computation. In [14] Rolf Pfeifer et al. emphasize that there are information-theoretic consequences of embodiment of agents in the world. They write: "By information theoretic implications of embodiment we mean the effect of morphology, materials and environment on neural processing, or better, the interplay of all these aspects. It turns out that materials, for example, can take over some of the processes normally attributed to control, a phenomenon that is called "morphological computation." There is no taxonomy of morphological computation yet, but we can roughly distinguish between sensor morphology taking over a certain amount of computation, similarly for shape and materials, and for the interaction with the environment." [14, p.23] These authors thus emphasize the aspect of physical embeddedness which all agents "suffer" that are embodied. This refers to the physical relation between the agent's components, their individual material properties, and their physical interactions with the environment. The authors argue strongly (and rightly) that physics holds advantages in terms of providing certain computation for free, computation that would otherwise require substantial effort. They cite space-variant vision [8] as one example where the inhomogeneous placement of light-sensitive cells supports computational vision tasks in sensory information processing. In another example, the authors discuss motor information processing by applying a physical spring as an artificial muscle. Summarizing, they 178 Morphological Computation – A Broad Perspective Feature Morphological Computation Amorphous Computing Elements individuals clones Organization heterogeneous homogeneous Interaction local and global local Space localized non localized Approach applied, concrete general, abstract Table 1: Contrasting Morphological Computation with Amorphous Computing. state: "The morphological computation in this case is the result of the complex interplay of agent morphology, material properties (in particular the "muscles," i.e. the springs), control (amplitude, frequency), and environment (friction, shape of the ground, gravity). Exploiting morphological computation makes cheap rapid locomotion possible because physical processes are fast and for free!" [14, p.25] In other words, morphological computation is very much concerned with the materials that do the computation, their physical properties, but also their form, i.e. spatial relationships of components to each other. Generally speaking, such systems are to be expected to be heterogeneous, and rather than being concerned with the abstraction of computational processes, these systems exist or are designed/evolved for a particular purpose or function. Efficiency criteria play a major role and abstraction has taken a backseat. However, as we shall see, the current notion of morphological computation falls short: It does not include the centre of information processing in a body, rather only input and output devices. Below we shall therefore propose a generalization of the concept of morphological computation that encompasses a central information processing unit. As we can see: A resounding contrarian view to amorphous computation which can be summarized in the Table 1. The Relation between Morphology, Space and Nature In a more recent publication by a group including Pfeifer [11] it is argued that the complexity and non-linearity provided by flexibly moving body parts of a robot while difficult to control is a source of computational power and therefore a desired feature. In a series of recent publications Gordana Dodig-Crnkovic has taken up the idea of morphological computation and built a framework called "info-computationalism" [5, 6, 7]. Within this framework of a new philosophy of computing, morphological computation is at the centre of all methods of natural computation. It is interesting to note that the relation between morphology, space and Nature was important already earlier in the history of the sciences. In the 19th century, it stood at the cradle of a new discipline: geography. It was Alexander von Humboldt, who, through a strike of genius, was able to connect the morphology of a landscape with its topology, and therefore with space, founding the discipline of geography at the intersection between physical sciences and the humanities [12]. Are we witnessing a similar phenomenon now 179 Morphological Computation – A Broad Perspective in regard to computation? Morphology always presupposes a notion of topology and space, otherwise shape and relation between parts could not be defined. Space, in turn, is intimately connected to Nature. Nature without space is unimaginable, therefore, we associate the beginning of natural systems, and indeed Nature, with the beginning of space as is reflected in the standard model of the universe [2]. Note that most of what has been hitherto considered computation in the literature does not require a notion of space, perhaps with the exception of cellular automata [4, 15]. Some have, however, argued in turn, that the universe is a calculating machine [9, 13], based on notions of cellular automata and thus based on the only model of spatial computation known at the time. Morphological Computation A Broad Perspective In recent years, computer scientists have realized that the world is more complex than what the notions of a Turing machine or a von-Neumann machine (a clever embodiment of the concepts of computation promoted by Turing) suggest. As Dodig-Crnkovic points out, these notions are typically closed system models, isolated boxes in which "computation" happens determined by a static input, a machine in defined states and an algorithm that should be left alone until a result emerges. The emphasis on the halting problem in classical theoretical computer science is a typical outcome of this thinking. Sure enough, if the algorithm doesn't halt, no defined result emerges, and the whole computation has a problem. Contrast that with what is now called data stream processing [3], a model of computation where input consists of continuous, unbounded, rapidly changing streams of data elements. Such a model of open, dynamically changing, massively parallel data streaming into a computational system is perhaps better suited to capture the interaction of a living organism with its environment than if we try to formulate algorithms for a closed systems. What it requires is a constantly running computational engine, because the data streams also never cease to flow. These open computational systems need to have a means to store relevant information extracted from the data stream in a finite storage. Despite the infinity of the stream, a way must be found to store information extracted from the stream in a finite memory. Most of the stream must leave the system after it has contributed to some computation and will have to be discarded ultimately, due to the impossibility of accumulating the content of the stream. In fact, a better metaphor would be to say that the computational engine must sit at the "shore" of the data stream and dip into it as required.1 1 This would mean the machine would merely observe the data stream, without "swallowing" it. This would entail the necessity of a translation of these observations into an internal representation of the machine, and would allow different machines to produce different representations/extracts/results from the same data stream, see [11] for a similar argument about the usefulness of such a property. In particular, this could open the door for a competitive evolutionary process to improve information extraction from the data stream, just like what happened with the evolution of brains. 180 Morphological Computation – A Broad Perspective Hauser et al. [11] make a similar argument as regards the "morphological computation with compliant bodies." They claim that for biological systems a mapping from input streams to output streams (which could be formalized by mathematical operators) is closer to real demands than a Turing machine. Our expectation is that this is a fundamentally different model of computation which requires attention from computer scientists for its potential to one day eclipse the Turing/von-Neumann paradigm. Such a fundamental shift will probably come about by the practical needs of applications. In the area of data streaming we are witnessing already the emergence of such needs . A few examples of typical data streaming applications are: • Network traffic streams • Financial data streams • Sensor network data streams • Weather observations • Measurements and observations from physical systems like particle accelerator data or telescope data • Sensor input streams for robot navigation • Activity streams in online computer games It is interesting to note that the idea of data streams was discussed first in the database community, before finally spilling over into other areas of computer science. The question originally was how to manage data streams, as they obviously constituted challenges to the storage capacity of database systems. So query systems for a data stream management system (DSMS) have been prevalent in researchers' minds [10]. While it is certainly important to study how to manage data streams, the more important question in our context is whether the principles of computation applied to streams of data would not need to be radically changed as well. I would answer this question in the affirmative. The solution to this problem is morphological computation in the sense that different centres of processing will feed on the data stream, and extract pieces of compressed information in a way prescribed by their type. Just as the brain consists of many localized centres of information processing of different types, all connected to each other and ultimately to sensors and effectors, so a computing machine could be constructed using localized centres specializing in different types of algorithms, all either connected to each other or, ultimately, to sensors and effectors. As an example, consider a soft computing machine consisting of localized centers, for instance, a machine learning centre, a genetic search engine, a fuzzy reasoning centre, a swarm intelligence device, and a group of neural networks. We use to think of these entities as different types of algorithms, but in morphological computation they each exist side by side, in a spatially configured computing machine. They are localized, 181 Morphological Computation – A Broad Perspective Figure 1: In this sketch, two soft computing machines (SC 1 and SC 2) dip into a transient data stream and copy information into their internal data representation. Subsequently, internal processing allows them to compute reactions that constitute their output. heterogeneous pieces of (possibly different, specialized) hardware and connect to each other. Input to this machine would be from a stream of data, and changes in processing of information would be effected not so much by changes to the different centres, but by different routing of the internal representation of the data stream and its higher-level extracts. We can see immediately, that this is the idea of morphological computation, this time, however, applied to the computation itself. Modern brains of higher-level organisms are organized precisely in this fashion. The human brain, for instance, has more than 50 nuclei, different subsystems of nerve cells discernible by their internal connectivity structure and the elements performing the computation. Major changes in information processing in brains are achieved not so much by reprogramming particular specialized nuclei, but by changing their connection to other nuclei and the introduction of new nuclei. In a similar manner, major changes in the soft computing machine sketched above could be achieved by introducing new centres for processing information, as well as by rerouting the information circulating in the system. Thus, programming would be a fundamentally different process. Sensor information streaming into the system would have to be ultimately discarded in favour of the extraction of highly compressed representations of these streams. Figures 1, 2 show a sketch of this idea. In general, this open model of computation is much more natural and could be argued to be the quintessence of embodied computation. It comes with another advantage: The possibility to allow a multitude of these machines to access the same data stream. Why would this be an advantage? It could be used to set up a competition between the different machines for the best quality of information extraction from the data stream. Assuming that all machines receive virtually the same input (a transient stream they 182 Morphological Computation – A Broad Perspective Figure 2: An example of a soft computing machine's internal organization: A number of modules (C1, C2, etc.) are connected, each extracting other (possibly contradictory information from the encoded data stream. Integration and disambiguation must be sought to present the outside world with a defined reaction to the stream. The general character of processing is feed-forward from input to output, but occasionally there might also be feedback connections, mostly within modules (not shown here). Encoding is such that (i) it does not disturb the data stream; (ii) it can be discarded within the system. There is continuous input and output to the machine. Some of the components might be memory devices that store some high-level representation of extracted information. copy), they have the same starting conditions for information processing. Assuming that there are different routings of information, and perhaps different ways of encoding information within a machine, the results of information extraction and ultimately the reaction of the machines to the data stream might well be diverse. In such a situation, competition (and perhaps an evolutionary process) could set in for improving coding, information routing and processing to allow for the extraction of the optimal amount of high-level information or "knowledge," as it might be legitimately called. Bibliography [1] H. Abelson, D. Allen, D. Coore, C. Hanson, G. Homsy, T. Knight Jr, R. Nagpal, E. Rauch, G. Sussman, and R. Weiss. Amorphous Computing. Communcations of the ACM, 43(5):74–82, 2000. [2] C. Bennett. Cosmology from start to finish. Nature, 440:1126 – 1131, 2006. [3] M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Cetintemel, Y. Xing, and S. Zdonik. Scalable distributed stream processing. In Proc. Conf. on Innovative Data Syst. Res, 2003. [4] E. Codd. Cellular automata. Academic Press, 1968. 183 Morphological Computation – A Broad Perspective [5] G. Dodig-Crnkovic. Significance of models of computation, from turing model to natural computation. Mind and Machines, 21:301–322, 2011. [6] G. Dodig-Crnkovic. Info-computationalism and morphological computing of informational structure. In P. L. Simeonov, L. S. Smith, and A. C. Ehresmann, editors, Integral Biomathics, pages 97–104. Springer, Berlin Heidelberg, 2012. ISBN 978-3642-28111-2. [7] G. Dodig-Crnkovic. The info-computational nature of morphological computing. In V. C. Mueller, editor, Philosophy and Theory of Artificial Intelligence, volume 5 of Studies in Applied Philosophy, Epistemology and Rational Ethics, pages 59–68. Springer, Berlin Heidelberg, 2013. ISBN 978-3-642-31674-6. [8] F. Ferrari, J. Nielsen, P. Questa, and G. Sandini. Space variant imaging. Sensor Review, 15:18–20, 1995. [9] E. Fredkin. Finite nature. In Proceedings of the XXVIIth Rencotre de Moriond, 1992. [10] L. Golab and M. T. Özsu. Issues in data stream management. SIGMOD Rec., 32 (2):5–14, June 2003. ISSN 0163-5808. [11] H. Hauser, A. Ijspeert, R. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological cybernetics, 105:355–370, 2011. [12] A. v. Humboldt. Kosmos, Entwurf einer physischen Weltbeschreibung. Cotta, Stuttgart, 1845. (Cosmos: A sketch of a physical description of the universe, translated by E.C.Otté, London, Henry G. Bohn, 1848). [13] K. Mainzer and L. Chua. The Universe as Automaton From Simplicity and Symmetry to Complexity. Springer, Berlin Heidelberg, 2012. [14] R. Pfeifer, F. Iida, and G. Gómez. Morphological computation for adaptive behavior and cognition. International Congress Series, 1291:22–29, 2006. [15] K. Zuse. Rechnender Raum. Vieweg und Sohn, Braunschweig, 1969. (Calculating Space, MIT Technical Translation AZT-70-164-GEMIT, Massachusetts Institute of Technology, Cambridge, MA, 1970). 184 Abstract: Tailoring the design of robot bodies for control purposes is implicitly performed by engineers, however, a methodology or set of tools is largely absent and optimization of morphology (shape, material properties of robot bodies, etc.) is lagging behind the development of controllers. This has become even more prominent with the advent of compliant, deformable or "soft" bodies. These carry substantial potential regarding their exploitation for control – sometimes referred to as "morphological computation" in the sense of offloading computation needed for control to the body. Here, we will argue in favor of a dynamical systems rather than computational perspective on the problem. Then, we will look at the pros and cons of simple vs. complex bodies, critically reviewing the attractive notion of "soft" bodies automatically taking over control tasks. We will address another key dimension of the design space – whether model-based control should be used and to what extent it is feasible to develop faithful models for different morphologies. Trade-Offs in Exploiting Body Morphology for Control Introduction It has become increasingly common to explain the intelligent abilities of natural agents through reference to their bodily structure, their morphology, and to make extended use of this morphology for the engineering of intelligent abilities in artificial agents, e.g. robots–thus "offloading" computational processing from a central controller to the morphology. These uses of morphology for explanation and engineering are sometimes referred to as "morphological computation." As we argue in detail elsewhere [21], only some of the characteristic cases that are embraced by the community as instances of morphological computation have a truly computational flavor. Instead, many of them are concerned with exploiting morphological properties to simplify a control task. This has been labeled "morphological control" in [8]; "mechanical control" could be an alternative label. Developing controllers that exploit a given morphology is only a first step. The space of possible solutions to a task increases dramatically once the mechanical design is included in the design space. At the same time, the dimensionality of the design problem grows exponentially. In this work, we want to take a close look at these issues. First, we will borrow the "trading spaces" landscape from [28] that introduces a number of characteristic examples and distributes them along a metaphorical axis from "informational computation" to "morphological computation." Second, we will argue in favor of a dynamical systems rather than computational perspective on the problem. Third, we will critically look at the pros and cons of simple vs. complex (highly dimensional, dynamic, nonlinear, compliant, deformable, "soft") bodies. Fourth, we will address another key dimension of the design space whether model-based control should be used and to what extent it is feasible to develop faithful models for different morphologies. We will close with an outlook into the future of "soft" robotics. Design "Trading Spaces" Pfeifer et al. [28] offer one possible perspective on the problem in Figure 1. In traditional robots – as represented by industrial robots and Asimo in the figure – , control is essentially confined to the software domain where a model of the robot exists and current state of the robot and the environment is continuously being updated in order to generate appropriate control actions sent to the actuators. In biological organisms, on the other hand, this does not seem to be the case: the separation between "controllers" and "controlled" is much less clear and behavior is orchestrated through a distributed network of interactions of informational (neural) and physical processes. Furthermore, there is no centralized neural control, but a multitude of recurrent loops from the lowest level (reflexes and pattern generators in the spinal cord) to different subcortical and cortical areas in the brain. At the same time, the bodies themselves tend to be much more complex in terms of geometrical as well as dynamical properties. This has motivated the design of compliant, tendon-driven robots like ECCE [36] and soft, deformable robots like Octopus (e.g., [16]). However, compared to humans or biological octopus, a 186 Trade-Offs in Exploiting Body Morphology for Control Figure 1: The design trading space. This figure illustrates the degree to which each system relies on explicit control or self-organization of mechanical dynamics. On the left-hand side of the spectrum, computer algorithms and commercial computers rely on physical self-organization at the minimum level, while towards the right-hand side, more embodied, more soft, and smaller-scale systems require physical interactions as driving forces of behaviors. The design goal then is to find a proper compromise between efficiency and flexibility, taking into account that a certain level of flexibility can also be achieved by changing morphological and material characteristics. (Figure and caption from [28]). comparable level of versatility and robustness in the orchestration of behavior has not yet been achieved in the robotic counterparts. In more restricted settings, the design and subsequent exploitation of morphology is easier, as the jumping and landing robot frog [22], the passive dynamic based walker, or the coffee balloon gripper demonstrate. The passive dynamic walkers [19] are a powerful demonstration that appropriate design of morphology can generate behavior in complete absence of software control. Yet, there is only a single behavior and the environmental niche is very narrow. The coffee-balloon gripper [4] employs a similar strategy, but achieves surprising versatility on the types of objects that can be grasped. Body designs that follow this guideline were also labeled "cheap designs" [26]. From Computational to Dynamical Systems Perspective on the Control Problem A general formulation of a control problem in control theory is making a dynamical system follow a desired trajectory. For our purposes, we will consider the cases where the dynamical system is physical the body of the agent; in control theory, this is the socalled plant. There are numerous control schemes and branches of control theory and the reader is referred to abundant literature on the topic (e.g., [3, 7, 15]). The performance of the controller can be evaluated on various grounds: precision of a trajectory with respect to a reference trajectory, or energy expenditure, for example. In addition, performance, 187 Trade-Offs in Exploiting Body Morphology for Control stability and robustness guarantees are required by industry. Control theory typically deals with the design of controllers that optimize these criteria. Some control schemes with appropriate cost functions will automatically result in minimal control actions and thus "optimize the contribution of the morphology." For example, Moore et al. [20] used Discrete Mechanics and Optimal Control to steer a satellite while exploiting its dynamics to the maximum. Carbajal [6] developed related methods for reaching, plus offered a formalization of the concept of "natural dynamics." Nevertheless, the plant is treated as fixed in these approaches. Yet, the properties of the physical body obviously have a key influence on the final performance of the whole system (plant + controller), which calls for including them into the design space. Therefore, including the body into the design space carries great potential. We feel that to address the problem of morphology simplifying control, a computational perspective does not provide the most appropriate framework. Instead, the dynamical systems description seems more appropriate for the following reasons: (i) It fits the informational and physical processes equally well; (ii) It copes with continuous (in time) streams of continuous input and output signals; (iii) It is already used by control theory. The concept of self-stabilization, for example, which is often cited in the "morphological computation community" is naturally explained from a dynamical systems standpoint. In the case of the passive dynamic walker or the jamming-based grippers cited above, the body is ingeniously contributing to its, perhaps primary, function: enabling physical behavior in the real world. While this is often interpreted in the "offloading sense" – the body design takes over computation from the brain (e.g., [25]) – we argue that the contribution of the body itself (and the interaction with the environment) is not computational in any substantial sense. In fact, calling it computation may allude to a much stronger thesis (at least in the classical notion of computation), namely that what is done by the interaction here could also be done by a conventional computer. And that is quite the opposite of what most proponents of the morphological stance want to say. Therefore, our claim is that exploiting morphology for control should be better handled in a non-computational context and that the dynamical systems framework is more appropriate than a classical computational one. Simple or Complex Bodies? The spirit of the morphological computation literature that follows the "offloading" or "trade-off" perspective, is that complex (highly dimensional, dynamic, nonlinear, compliant, deformable, "soft" ) bodies are advantageous for control because they can take over the "computation" that a controller would otherwise have to perform (e.g., [9, 10, 25] or [5] explicitly in Figure 1). Complex nonlinear bodies give rise to more complex dynamical landscapes where the location of attractors can facilitate the performance on a given task. This view is in stark contrast to the views prevalent in control theory. There, linear time-invariant systems are the ideal plants to control. Solutions for nonlinear systems are much more difficult to obtain and they often involve a linearization of the system of 188 Trade-Offs in Exploiting Body Morphology for Control some sort. In fact, human-like bodies are a nightmare for control engineers ([29] is an interesting case study) and highly complex models and controllers would be required. What would be an ideal body then? And, does a complex body imply simple or complex control? Recent attempts at quantifying the amount of morphological computation shed more light on this issue. Zahedi and Ay [37] propose two concepts for measuring the amount of morphological computation by calculating the conditional dependence of future world states (or body state) on previous world states and action taken by the agent. According to Concept 1, the amount of morphological computation is inversely proportional to the contribution of the agent's actions to the overall behavior. Concept 2 equates the amount of morphological computation with the contribution of the world to the overall behavior. From a robotic perspective, optimizing for morphological computation, Concept 1 is equivalent to systems that cannot be controlled by actions of the robot; Concept 2 will give rise to systems with strong "body dynamics" or "natural dynamics" (see e.g., [14] or [6] for a formal definition). A body with strong internal dynamics (Concept 2), resisting control actions (Concept 1), will not be a product of typical engineering designs. Yet, if it is possible to design the body to assist in task performance, it is certainly of advantage. The passive dynamic walker is an extreme case; however, underactuated designs featuring active and passive degrees of freedom are also abundant. Rückert and Neumann [31] study learning of optimal control policies for a simulated 4-link pendulum which needs to maintain balance in the presence of disturbances. The morphology (link lengths and joint friction and stiffness) is manipulated and controllers are learned for every new morphology. They show that: (1) for a single controller, the complexity of the control (as measured by the "variability" of the controller) varies with the properties of the morphology: certain morphologies can be controlled with simple controllers; (2) optimal morphology depends on the controller used; (3) more complex (time-varying) controllers achieve much higher performance than simple control across morphologies. In summary, the performance on a task will always depend on a complex interplay of the controller, body, and environment: taking out the controller is just as big a mistake as taking out the body was. The tasks that can be completely solved by appropriate tuning of the body, like passive dynamic walking, are the exception rather than the rule. A controller will thus be needed too. A complex body may have the potential to partially solve certain tasks on its own; yet, it may present itself as difficult to control, model (if the controller is relying on models), design, and manufacture. An optimal balance thus needs to be found. For that, however, new design methodologies that would encompass complex cost functions (performance on a task, versatility, robustness, costs associated with hardware whose parameters can be manipulated etc.) are needed. Hermans et al. [11] very recently proposed such a method that uses machine learning to optimize physical systems; an approximate parametric model of the system's dynamics and sufficient examples of the desired dynamical behavior need to be available though which leads us to the next section. 189 Trade-Offs in Exploiting Body Morphology for Control With or Without a Model? Including the parameters of the body into the design considerations may give rise to better performance of the whole system; these may be solutions involving a simpler controller, but also solutions that were previously unattainable when the body was fixed. Following the dynamical system's perspective, [8] provide an illustration of the possible goals of the design process: (1) To design the physical dynamical system such that desired regions of the state space have attracting properties. Then it is sufficient to use a simple control signal that will bring the system to the basins of attraction of individual stable points that correspond to target behaviors. (2) More complicated behavior can be achieved if the attractor landscape can be manipulated by the control signal. If a mathematical formulation of the controller and the plant is available, this design methodology can be directly applied. The first part is demonstrated by McGeer [19] on the passive dynamic walker: The influence of scale, foot radius, leg inertia, height of center of mass, hip mass and damping, mass offset, and leg mismatch is evaluated. In addition, the stability of the walker is calculated. Recently, Jerrold Marsden and his coworkers presented a method that allows for co-optimization of the controller and plant by combining an inner loop (with discrete mechanics and optimal control) and an outer loop (multiscale trend optimization). They applied it to a model of a walker and obtained the best position of the knee joints ([24] – Ch. 5). However, typical real-world agents are more complex than simple walkers. Holmes et al. [12] provide an excellent dynamical systems analysis of the locomotion of rapidly running insects and derive implications for the design of the RHex robot. Yet, they conclude that "a gulf remains between the performance we can elicit empirically and what mathematical analyses or numerical simulations can explain. Modeling is still too crude to offer detailed design insights for dynamically stable autonomous machines in physically interesting settings." Hermans et al. [11] similarly note that applying their method to robotics, which is known to suffer from lack of accurate models, is a challenge. The modeling and optimization of more complicated morphologies like compliant structures is nevertheless an active research topic (e.g., Wang [35] and other work by the author). The second point of Füchslin et al. [8] achieving "morphological programmability" by constructing a dynamical system with a parametrized attractor landscape remains even more challenging though. One of the merits of exploiting the contributions of body morphology should be that the physical processes do not need to be modeled, but can be used directly. However, without a model of the body at hand, several body designs need to be produced and together with the controller tested in the respective task setting. The design space of the joint controller-body system blows up and we may be facing a curse of dimensionality. This is presumably the strategy adopted by the evolution of biological organisms that could cope with the enormous dimensionality of the space. In robotics, this has been taken up by evolutionary robotics [23]. The simulated agents of Karl Sims [32] demonstrate that co-evolving brains and bodies together can give rise to unexpected solution to problems. More recently, with the advent of rapid prototyping technologies, physics-based simulation could be complemented by testing in real hardware [18]. Yet, a 190 Trade-Offs in Exploiting Body Morphology for Control "reality gap" always remains between simulated and real physics and searching for optimal designs in hardware directly is very costly (e.g., [13]). The design decisions which parameters to optimize are based on heuristics and a clear methodology is still missing. Furthermore, with the absence of an analytical model of the controller and plant, no guarantees on the system's performance can be given. Outlook In terms of applications, the most relevant area where exploitation of morphology is and will be the key is probably robotics, and in particular soft robotics (see [2, 27, 28, 34] and the first issue of the Soft Robotics Journal [33]). "Soft" robots break the traditional separation of control and mechanics and exploit the morphology of the body and properties of materials to assist control as well as perceptual tasks. Pfeifer et al. [27] even discuss a new industrial revolution. Appropriate, "cheap", designs lead to simpler control structures, and eventually can lead to technology that is cheap in a monetary sense and thus more likely to impact on practical applications. Yet, a lot of research in design, simulation and fabrication is needed (see [17] for a review). The area of soft robotics and morphological computation seems to be rife with different trading spaces [28]. As we move from the traditional engineering framework with a central controller that commands a "dumb" body toward delegating more functionality to the body, some convenient properties will be lost. In particular, the solutions may not be portable to other platforms anymore, as they will become dependent on the particular morphology and environment (the passive dynamic walker is the extreme case). The versatility of the solutions is likely to drop as well. To some extent, the morphology itself can be used to alleviate these issues if it becomes adaptive. Online changes of morphology (like changes of stiffness or shape) thus constitute another tough technological challenge (see also project LOCOMORPH [1]). Completely new, distributed control algorithms that rely on self-organizing properties of complex bodies and local distributed control units will need to be developed (the tensegrity structure controlled by a spiking neural network [30] is a step in this direction). Acknowledgements M.H. was supported by the Swiss National Science Foundation Fellowship PBZHP2147259. M.H. also thanks Juan Pablo Carbajal for fruitful discussions and pointers to literature. Both authors thank the EUCogIII project (FP7-ICT 269981) for making us talk to each other. Bibliography [1] Project LOCOMORPH. URL http://locomorph.eu/. FP7-ICT-231688. 191 Trade-Offs in Exploiting Body Morphology for Control [2] A. Albu-Schaffer, O. Eiberger, M. Grebenstein, S. Haddadin, C. Ott, T. Wimbock, S. Wolf, and G. Hirzinger. Soft robotics. Robotics Automation Magazine, IEEE, 15 (3):20 –30, September 2008. ISSN 1070-9932. doi: 10.1109/MRA.2008.927979. [3] K. Aström and R. M. Murray. Feedback Systems: An Introduction for Scientists and Engineers. Princeton University Press, 2008. [4] E. Brown, N. Rodenberg, J. Amend, A. Mozeika, E. Steltz, M. R. Zakin, H. Lipson, and H. M. Jaeger. From the cover: Universal robotic gripper based on the jamming of granular material. Proc Natl Acad Sci U.S.A., 107(44):18809–18814, Nov 2010. doi: 10.1073/pnas.1003250107. [5] K. Caluwaerts, M. D'Haene, D. Verstraeten, and B. Schrauwen. Locomotion without a brain: physical reservoir computing in tensegrity structures. Artificial Life, 19(1): 35–66, 2013. [6] J. P. Carbajal. Harnessing Nonlinearities: Behavior Generation from Natural Dynamics. PhD thesis, University of Zurich, 2012. [7] A. Emami-Naeini, G. F. Franklin, and J. D. Powell. Feedback control of dynamic systems. Prentice Hall, 2002. [8] R. Füchslin, A. Dzyakanchuk, D. Flumini, H. Hauser, K. Hunt, R. Luchsinger, B. Reller, S. Scheidegger, and R. Walker. Morphological computation and morphological control: steps towards a formal theory and applications. Artificial Life, 19 (1):9–34, 2013. [9] H. Hauser, A. Ijspeert, R. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological Cybernetics, 105:355–370, 2011. [10] H. Hauser, A. Ijspeert, R. Füchslin, R. Pfeifer, and W. Maass. The role of feedback in morphological computation with compliant bodies. Biological Cybernetics, 106: 595–613, 2012. [11] M. Hermans, B. Schrauwen, P. Bienstman, and J. Dambre. Automated design of complex dynamic systems. PLOS ONE, 9(1):e86696, 2014. [12] P. Holmes, R. J. Full, D. Koditschek, and J. Guckenheimer. The dynamics of legged locomotion: Models, analyses and challenges. SIAM Review, 48(2):207–304, 2006. [13] G. Hornby, S. Takamura, J. Yokono, O. Hanagata, T. Yamamoto, and M. Fujita. Evolving robust gaits with AIBO. In Robotics and Automation, 2000. Proceedings. ICRA '00. IEEE International Conference on, volume 3, pages 3040 –3045 vol.3, 2000. doi: 10.1109/ROBOT.2000.846489. [14] F. Iida, G. Gómez, and R. Pfeifer. Exploiting body dynamics for controlling a running quadruped robot. In Proceedings of the 12th Int. Conf. on Advanced Robotics (ICAR05)., pages 229–235, Seattle, U.S.A., 2005. 192 Trade-Offs in Exploiting Body Morphology for Control [15] D. Kirk. Optimal control theory: an introduction. Dover Publications, 2004. [16] C. Laschi, M. Cianchetti, B. Mazzolai, L. Margheri, M. Follador, and P. Dario. Soft robot arm inspired by the octopus. Advanced Robotics, 26(7):709–727, 2012. [17] H. Lipson. Challenges and opportunities for design, simulation, and fabrication of soft robots. Soft Robotics, 1:21–27, 2013. [18] H. Lipson and J. Pollack. Automatic design and manufacture of robotic lifeforms. Nature, 406(6799):974–978, 2000. [19] T. McGeer. Passive dynamic walking. The International Journal of Robotics Research, 9(2):62–82, 1990. [20] A. Moore, S. Ober-Blöbaum, and J. Marsden. Trajectory design combining invariant manifolds with discrete mechanics and optimal control. Journal of Guidance, Control, and Dynamics, 35(5):1507–1525, 2012. [21] V. Müller and M. Hoffmann. Is morphological computation really computation? Contributions of the body to cognition and control. [submitted to Artificial Life]. [22] R. Niiyama, A. Nagakubo, and Y. Kuniyoshi. Mowgli: A bipedal jumping and landing robot with an artificial musculoskeletal system. In IEEE Int. Conf. Robotics and Automation (ICRA), pages 2546–2551. IEEE, 2007. [23] S. Nolfi and D. Floreano. Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines, volume 26. MIT press Cambridge, 2000. [24] D. Pekarek. Variational methods for control and design of bipedal robot models. PhD thesis, California Institute of Technology, 2010. [25] R. Pfeifer and J. Bongard. How the body shapes the way we think: a new view of intelligence. MIT Press, Cambridge, MA, 2007. ISBN 0-262-16239-3. [26] R. Pfeifer and C. Scheier. Understanding Intelligence. MIT Press, 1999. [27] R. Pfeifer, M. Lungarella, and F. Iida. The challenges ahead for bio-inspired 'soft' robotics. Communications of the ACM, 55(11):76–87, 2012. [28] R. Pfeifer, H. Marques, and F. Iida. Soft robotics: the next generation of intelligent machines. In Proc. 23rd Int. Joint Conf. on Artificial Intelligence, pages 5–11. AAAI Press, 2013. [29] V. Potkonjak, B. Svetozarevic, K. Jovanovic, and O. Holland. The puller-follower control of compliant and noncompliant antagonistic tendon drives in robotic systems. Int. Journal Advanced Robotic Systems, 8(5):143–155, 2011. [30] J. Rieffel, F. Valero-Cuevas, and H. Lipson. Morphological communication: exploiting coupled dynamics in a complex mechanical structure to achieve locomotion. Journal of the Royal Society Interface, 7(45):613–621, 2010. 193 Trade-Offs in Exploiting Body Morphology for Control [31] E. Rückert and G. Neumann. Stochastic optimal control methods for investigating the power of morphological computation. Artificial Life, 19:115–131, 2013. [32] K. Sims. Evolving 3D morphology and behavior by competition. Artificial Life, 1 (4):353–372, 1994. [33] B. Trimmer. A journal of soft robotics: Why now? Soft Robotics, 1:1–4, 2013. [34] D. Trivedi, C. Rahn, W. Kier, and I. Walker. Soft robotics: Biological inspiration, state of the art, and future research. Applied Bionics and Biomechanics, 5(3):99–117, 2008. [35] M. Y. Wang. A kinetoelastic formulation of compliant mechanism optimization. Journal of Mechanisms and Robotics, 1(2):021011, 2009. [36] S. Wittmeier, C. Alessandro, N. Bascarevic, K. Dalamagkidis, D. Devereux, A. Diamond, M. Jäntsch, K. Jovanovic, R. Knight, H. Marques, et al. Toward anthropomimetic robotics: Development, simulation, and control of a musculoskeletal torso. Artificial Life, 19(1):171–193, 2013. [37] K. Zahedi and N. Ay. Quantifying morphological computation. Entropy, 15(5): 1887–1915, 2013. 194 Abstract: Morphological computation can be loosely defined as the exploitation of the shape, material properties, and intrinsic dynamics of a physical system in order to improve the efficiency of a computation. Morphological control is the application of morphological computing to a control task. In this paper, we discuss possible applications of the concept of morphological control to problems in medicine and clinical therapy. We motivate our conviction that the emergent dynamical systems studied in the various branches of the omics-sciences should be analyzed from the perspective of a broader notion of control engineering and from what has been called biochemical IT (by "omics"-sciences we mean genomics, proteomics, metabolomics, glycomics etc. For the interpretation relevant to this paper, see [23]). This conviction is illustrated by three hypotheses. The first one claims the often experienced age-related loss of control over basic movement patterns, such as walking, are not (only) caused by a degradation of the nervous system. Of similar importance is also the loss of morpho-computational power by changes of the mechanical properties of the patient's body. If true, this hypothesis suggests novel types of support systems, one of which we presented in earlier works. The second hypothesis regards models that describe the dynamics of a population of tumor cells. We discuss a way to optimize synergistic treatments (combination of hyperthermia and radiotherapy) that is based on the assumption of a low-dimensional control system. Finally, the third hypothesis presents a morphological implementation of Matzinger's mechanism of danger signals in immunology. The model we present is inspired by studies in the origin-of-life research, where possibilities to control molecular parasitism are discussed. The article closes with the discussion of a scientific roadmap for the application of morphological control in the field of medicine. Morphological Control as Guiding Principle in Physiology and Medical Applications Introduction The control schemes of living or life-like systems differ strongly from what one is used from conventional engineering and control theory. One (of several) fundamental differences is found in an only gradual distinction between hardand software. To understand the mechanisms enabling this blurring as well as how to exploit it is regarded as a major route towards artificial devices and procedures exhibiting the properties usually attributed to living systems: robustness, adaptivity and self-organized, decentralized control. The interplay between physics and/or chemistry on one side and aspects of control or information processing on the other side has been investigated under a couple of umbrellas, e.g. embodied intelligence [28, 29], biochemical IT (see the EU-funded project COBRA), soft robotics and morphological computation (visit SoftRobot2013 to learn more about the continuing series of conferences on morphological computation) or, more specifically, morphological control [10]. A, of course oversimplified, way to capture what the application of the concept of morphological control in engineering means is to say that it turns the matter (forming an artifact) as a subject of control into matter being a subject that exerts control. We emphasize, that when speaking of morphology, we always refer to shape as well as to properties such as elasticity, friction coefficients and the like. In this essay, we hypothesize about morphological control as a guiding principle for understanding and handling medical problems. Our basic assumption is that a physiological system, be it a cell, an organ or even a limb, is governed firstly by neural or genetic control signals and secondly by its intrinsic dynamics. These dynamics is characterized by an attractor landscape. Explicit neural or genetic control chooses the basin of attraction and forces the system to move into it. Once residing in the proper basin, the intrinsic dynamics of the system takes over the fine-tuning and stabilization against small perturbations. Central for the approach of morphological control is the assumption that the "division of labor" between neural or genetic control on one side (sometimes called "explicit control") and the physico-chemical dynamics ("implicit control") on the other side did not evolve in arbitrary fashion, but is determined by an optimality criterion that is not yet fully understood. This idea, to our knowledge first explicitly expressed by Rolf Pfeifer from the AILab, Univ. Zurich, can then be utilized for analyzing physiological control systems: Given the (yet to be formulated in generality) optimality criterion and given knowledge about either explicit or implicit control, we can draw conclusions about its respective partner. And particularly important for medical applications is: If we had a measure to what degree the balance between explicit and morphological control differs from the theoretical optimum, we would have a diagnostic tool at hand. More practically, we claim that insights from morphological control are of value for the development of technical devices and pharmaceutical approaches to alter attractor landscapes of physiological systems. The technical devices are primarily thought to be hardware devices either equipped with a control scheme that is itself based on a hybrid of explicit and implicit control or that supports or complements respective physiological control mechanism. Therapies based on such devices harvest the lessons learnt from the application of morphological control in engineering, especially in the field of robotics. Specifically, these lessons serve to define therapeutic protocols based on the assumption 196 Morphological Control as Guiding Principle in Physiology and Medical Applications Figure 1: The concept of morphological control. A) Conventional control approaches exert control based on a (as far as possible) comprehensive representation of the state of a system. This representation is used as input for a conventional computer that determines commands which are sent to actuators. Physical dynamics not covered by the representation are reduced as far as possible. B) In contrast, morphological control aims at delegating control tasks to the physical dynamics of the system to be controlled. As discussed in the text, such a system may, under suitable circumstances, be run by a feed-forward neural network, although this is of course no requirement. The thickness of the arrows indicates metaphorically the band width of the information flow. that physiological dynamics can be understood in terms of dynamic attractors, stochastic switching between basins of attraction, etc. Pharmaceutical approaches profiting from morphological control are based on similar ideas about attractors, though these attractors are those of chemical instead of mechanical systems. In [10] we detailed our definition of morphological control and made steps toward a formalization; here, we give a brief summary. In Figure 1, we compare conventional and morphological control using a robot as an example. In case of conventional control (Figure 1A), the physical state of the robot is determined by a small number of parameters such as angles or lengths and their respective rates of change (velocities). These parameters are measured by sensors and the observed values are used for the construction of a virtual representation of the robot based on a kinematic model. This virtual representation is completely abstract and processed by a digital computer in order to obtain directives which then are used to command actuators. Given that the free parameters of the robot can only be determined with finite accuracy and with a finite sampling rate, the control unit has to deal with "physical noise". Thus, to keep the computation as simple and reliable as possible, robots are designed as to minimize the undesirable influence of the morphology seen as the source of physical noise. One strategy is to keep the number of degrees of freedom as low as possible and fluctuations as small as possible. From a physical point of view, this is one of the reasons why most robots are heavy and stiff. Under these conditions, a limited number of system parameters can give an account of the state of the robot, e.g. its posture and velocities that is sufficiently complete and accurate to allow effective control. Morphological control pursues an alternative strategy, described in Figure 1B. Now the control system is designed to exploit the robot's natural dynamics instead of dampening them. It is no longer necessary to represent the complete state of the robot in the control unit, it is sufficient to know in which basin of attraction it 197 Morphological Control as Guiding Principle in Physiology and Medical Applications actually resides. As a consequence, robots with morphological control can be constructed from soft materials – "soft" implying many internal degrees of freedom. We emphasize that the choice of a feed-forward neural network as the instance exerting explicit control in Figure 1B happened with deliberation. In fact, in [17] Hauser et al. were able to demonstrate that a physical body equipped with a feed-forward neural network and a sufficiently rich mechanical dynamics can serve as a powerful computational device. In this article, however, we speak of dynamical systems and their respective attractor landscapes. For biological systems, the fact is central that the system itself may be capable of changing its attractor landscape (e.g. by learning and training) and interaction with/adaptation to the environment. Our views on these aspects are discussed in [10]. In this essay, we speak of dynamical systems as if their attractor landscape were static. We point out that this is a gross oversimplification which, however, in some cases can be justified because a parameterized attractor landscape with time-dependent parameters can be understood as a dynamical system with static attractor landscape but additional degrees of freedom. Important in our context is the question whether the dynamics of the parameters is controlled by the intrinsic dynamics of the system or by neural/genetic control. In the former case, a dynamical systems perspective is fully justified, the latter case requires a treatment as it is e.g. given in [17] and [18]. This essay is organized as follows. After this introduction, we present a number of hypotheses about topics in medicine where morphological control may provide valuable insights and/or the framework for novel therapeutic schemes. The first two hypotheses discuss the underlying assumptions of own research in the field of prosthetics (hypothesis 1) and in the optimization of therapies in oncology (hypothesis 2). This work has already been presented in [10], [32] and [33]. In this essay, we discuss open points with respect to the paradigm of a dynamic attractor landscape. The third hypothesis is more speculative and relates to questions in immunology. What we present is not necessarily new or surprising, given a background in control theory or engineering (see the influential article by Ahn et al. [2]). However, we made at various occasions the experience that the concept of an attractor landscape is not always familiar to medical practitioners, especially not with respect to limit cycles. Another experience we regularly made is that the engineer's or physicist's view of how to calibrate a therapy does not account for the complexities of the human physiology and imperative necessity of acting in a manner safe for the patient. We are aware that the hypotheses we present need much more scrutiny in order to be established. Having said this, we present a roadmap for the application of morphological control in medicine in which some of the presented claims may serve either as milestones or examples for implementations at the end of this essay. Hypothesis 1: Aging Reduces Morpho-Computational Capabilites A usual consequence of aging is the gradual loss of the capability to control complex movement patterns. Following the paradigm of embodied intelligence [28], we start with 198 Morphological Control as Guiding Principle in Physiology and Medical Applications the assumption that part of this control is executed by the body in the sense that the physical dynamics of our limbs stabilize oscillatory patters such as walking, thereby facilitating the control task of the nervous system. One cause for the observed loss of control is reduced sensory and neural performance. The notion of morphological control suggests a further explanation. Aging changes the mechanical properties of the body, for example by changing elasticity of sinews and ligaments, skeletal deformations, etc. This change alters the body's attractor landscape and thereby may well reduce its ability to contribute to control tasks, e.g. because of less well expressed basins of attraction that are easier to perturb and exhibit longer transients (Of course, by "aging" we refer to processes that occur after the normal development in childhood.) Viewed from this perspective, the difficulties many elderly people experience in controlling their movements are not caused (at least not exclusively) by poorer performance of their brains but by lack of computational / control support from their bodies. This interpretation is in accordance with the interpretation of aging as a loss of complexity. Lipsitz discusses this view in a broader context in [25]; the perspective of morphological computation focusses on the change of embodied intelligence caused by the aging process and technical possibilities to regain it. Conventionally, loss of control over movement patterns is addressed by support systems such as walking frames that make this control obsolete. In [10], we present a different approach. Based on a novel type of inflatable structure ("tensairity") that even can be actuated [26], we discuss an approach for a support system that does not intend to keep the movements of a patient in defined tracks or even drives the movements but changes the mechanical properties of body parts (in the system under development: the knee joint) in such a manner that the intrinsic dynamics of the legs resembles the ones of a younger person. For further discussion about this approach, see also the article by Hunt and Fang in this volume. Clinical evidence that simple technical means can "rejuvenate" at least mechanical aspects of the human body is given by the experiments reported in [16] and [30]. These authors investigate the effect of vibrations applied to the soles of the feet of elderly patients and report a remarkable improvement with respect to parameters measuring the quality of balance. Although we aim at mechanical (morphological) properties and not sensory capabilities, these experiments pursue a strategy similar to ours. Vibrations seem to improve the quality of the nervous signals sent to the brain; a possible explanation for this phenomenon could be stochastic resonance [6]. Once, the elderly brain (or to be precise, the brain in an elderly body) obtains improved input, its output leads to a performance comparable to that of a person at the height of his or her abilities, at least with respect to certain specific classes of tasks. The calibration of such a system, i.e. the definition of an appropriate therapy scheme that accustoms the patient to the support system and at the same time allows tuning parameters of the system such that the system optimally matches with the mechanical properties of the patient's body, is a challenge. We see here a chance to collaborate with clinical practitioners as well as with physiotherapists. It is the task of the engineers to communicate concepts such as attractor landscape in a manner that is acceptable for experts from non-technical fields and to support the design of efficient training schemes 199 Morphological Control as Guiding Principle in Physiology and Medical Applications such that the described support devices can be used by a wide range of patients. Hypothesis 2: If Cells are Subject to Morphological Control, Oncology can be Equipped with New Tools Most applications of morphological control take place in the realm of robotics, i.e. macroscopic systems governed by classical mechanics. As mentioned in the introduction, a major design principle in robotics is to keep the number of degrees of freedom as small as possible, resulting in stiff and heavy robots. This approach is recently questioned by what is called "soft robotics," see [21]. The investigated structures are deformable and, therefore, in principle equipped with infinitely many degrees of freedom, though only those modes related to low energies are in fact important. These structures differ fundamentally from the "soft matter" cells are made off. Chemical process management taking place in the eukaryotic cell is different in so far as its dynamics are mostly ruled by statistical mechanics including chemical kinetics. The interplay of processes on different length scales, ranging from molecular interactions over supra-molecular selfassembly (e.g. the construction of the cytoskeleton or the formation of clathrin-cages in receptor-mediated endocytosis) to the dynamics of bio-membranes includes phenomena not covered by the formalism of classical mechanics, e.g. phase transitions. Despite such differences, the underlying hypothesis of morphological control remains a valid approach: The physico-chemical system is governed by dynamics that can be described in terms of attractors, whereby these dynamics are in general robust against perturbations. Specific signals may trigger a jump from one basin of attraction into another one. The reactions belonging to a basin of attraction determine the chemical output, the attractor being a fixed point, a limit cycle or even a strange attractor. Genetic control is complemented by morphological control; oversimplified one can say that the former (genetics) dictates what and when something is done, the latter (chemical dynamics) determines the details of the way how it is done. Understanding the attractor landscape, at least relevant parts or a projection of it, may provide several types of insights. First, in the causes of diseases: one may be able to relate symptoms to changes in the attractor landscape (a phenomenological approach) but also find the underlying causes of such a change. The second and probably more important type of insights is related to possible cures. Once we know how the attractor landscape should be structured and how it actually looks in case of a specific disease, we can think of ways how to reshape the attractor landscape if it is distorted by some form of degradation. The relevant issue here is that in order to reshape an attractor landscape, we do not necessarily have to mend the damage that lead to the deformation, but only to counterbalance it with whatever means are appropriate. The approach is comparable to the one of the support system for patients with movement impairments, where we do not intend to replace stiffened sinews (or whatever causes the change of the mechanical properties of patient's body), but where we aspire to re-establish its original dynamics by applying external means. The dynamical systems perspective is of course by no means new. We give some examples, emphasizing that this choice is entirely subjective and by no means an overview. 200 Morphological Control as Guiding Principle in Physiology and Medical Applications Mathematical models of protein sorting analyze the processes underlying receptor mediated endocytosis in terms of dynamical systems [19]. Knox embeds this approach in a multi-level assessment of cancer, regarded as a complex disease [23]. Vilani et al. were even able to address the issue of cell differentiation, thereby going much beyond chemical kinetics [34]. Systems medicine [4] and systems pathology [7] may be regarded as descendants of systems biology, with a specific focus on malfunctions of complex physiological systems and the resulting diseases and their respective symptoms. We regard our application of the concept of morphological control to processes on the cellular level as part of these endeavors. In studies such as [32], where a model of the tumor response on synergistically combined hyperthermia and radio therapy is presented, we aim at using a phenomenological model for predictive purposes instead of asking for the molecular causes of specific dynamics. This is done in the spirit of computational medicine [36]. The family of models we analyzed is given by a model template: dN1 dt = f(N1, N2, ..., ), (1) dN2 dt = g(N1, N2, ..., ), (2) d  dt = R  h( ). (3) where N1 denotes the number of tumor cells which are not damaged by heat or radiation, N2 gives the number of tumor cells with lethal or sub-lethal damage, the dose rate R represents the absorbed radiation energy per mass, and   stands for the dose equivalent. The ellipsis are a placeholder for potential further parameters. For an explanation of these parameters, see [32]. In this setting, functions f and g determine the rates of damage induction, repair and cell killing on the population level. The function h describes the average repair of cellular damage. Specific choices for the functions f, g and h lead to different types of models, as described in [32]. These choices are justified by explicit assumptions about cellular processes, e.g. the repair kinetics. In previous work, Scheidegger et al. reported two types of models, which are able to reproduce biological observations. We do not intend to discuss the various specific models resulting from the model template here in this article. Instead, we want to draw the attention towards the observation that a low-dimensional parametrization as in Eqs. 1 is capable of explaining the dynamics of a system at least to some degree of accuracy. The whole intra-cellular dynamics is captured by only one parameter, namely   . The biological dose equivalent   is assumed to be proportional to the average number of unrepaired sub-lethal lesions per a single cell. These lesions are produced at a rate proportional to the dose rate R and reduced by cellular repair processes, the latter being modelled by a function h( ). The population dynamics of tumor cells is accessible by experiment. The function h( ) models cell repair induced by damage; repair is of course not desired in case of a therapy (but for healthy tissues). If h( ) is a linear function, the potential for therapy optimization is limited; a linear repair mechanism just means that lesions can be recognized and mended by some local cellular mechanism. However, recent experiments indicate h( ) 201 Morphological Control as Guiding Principle in Physiology and Medical Applications to be more complex [24]. A non-linear h( ) may indicate a more sophisticated control protocol. Such a protocol may or may not be governed by system properties. If the former holds, we may find ways of interfering with the system as a whole and disturb repair (of tumor cells) in an optimized way. There are various types of interference, the one we have in mind is a physical one: In the case of hyperthermia-radiotherapy, radiation is complemented by application of heat (hyperthermia). The hope is that if h( ) can be identified as a systemic mechanism at least partially controlled by the physics of the cell, optimized application of heat may offer a way to increase the efficiency of conventional radiation therapy. The goal is to determine how synergistic physical interventions (heat and radiation) can be combined such that healthy cells do not leave their basin of attraction whereas malignant cells die. Hypothesis Three: Morphological Check Sums In our third hypothesis, we claim aspects of morphological control to be of potential relevance for a fundamental problem in immunology. We relate findings obtained in the study of the origin of life to problem settings as they occur when the self has to be discriminated from the non-self, or better, the "dangerous" from the "non-dangerous" [27]. We point out that our claim is largely hypothetical and we will not give a detailed account of molecular mechanisms. One possible scenario for the origin of life assumes the existence of a combinatorially large family F of chain molecules, some of these molecules exhibiting catalytic functionality. By "combinatorial" we mean that the chain molecules are composed of different monomers. These monomers can be combined in arbitrary sequences; a prominent example is RNA. A generic non-catalytic member of this family of chain molecules shall be denoted by X . The crucial assumption is now that at least one sequence in F , in the latter called R, is equipped with a catalytic functionality that results in the replication of a general member Y 2 F (be Y itself a catalyst or not) according to: R+ Y ! R+ Y + Y 0 (4) (the dash indicates that the replication can either happen in a faithful or erroneous manner). Such a process is less speculative than it may look on a first glance; for RNAbased replication see e.g. [22]. The basic idea behind this scenario is that starting from a population of R which maintains itself by reproduction, further types of functional sequences are added by evolution and become part of an autocatalytic network. There are several problems with this approach; the first one is certainly the highly improbable appearance of the first population of R . But besides initiation, there is a further, fundamental issue that affects all self-replicating systems: In order to be evolvable, the replication must be non-exclusive or "generous" in the sense that R is able to catalyze the replication of a large class of chain molecules belonging to R and not only those with the same sequence as R . Discussions about complementary strands and copying from DNA to RNA aside, the RNAand DNA polymerases are in essence examples of generous replicators. If R were capable of discriminatingR 's from non-R's, a self-replicating population of 202 Morphological Control as Guiding Principle in Physiology and Medical Applications molecules could perhaps be maintained, provided sufficient supply of monomers, energy, etc., but no new functionality could be added. If R instead is a general replicase in the sense of Equation 4, then another problem occurs. Replication of chain molecules is prone to errors, i.e. mutations. As a consequence, a non-functional chain molecule X will be produced from time to time: R+R ! R+R+X. (5) If resources (available monomers, energy etc.) are limited and the replication takes place in a well-stirred (spatially homogeneous) reactor, the population eventually will break down because of "molecular parasitism," the accumulation of chain molecules that are copied and hence consume monomers but do not contribute to the reproduction functionality of the whole population. The underlying mechanism and a number of important phenomena, in our context especially important is the "error threshold," have been analyzed e.g. in [8]. Note that despite the fact that the possibility of erroneous replication eventually may destroy a replication system, it is also a prerequisite for evolution: Only if occasionally a novel sequence X is produced (which in some rare cases carries a catalytic functionality), new functions can be added to a replication system. Several investigations analyzed possible ways to control molecular parasitism. One approach is pattern formation in spatially heterogeneous reaction environments. This approach has been pioneered by Boerlijst and Hogeweg in [5]. Further work along this route include reaction diffusion systems with more complex reaction mechanism [11], the effect of complex formation [9]. Another mechanism for the control of parasites has been presented by Altmeyer et al. in [3]. The basic idea there relies on the fact that chain molecules, proteins as well as RNA, attain secondary and tertiary structures. The model presented by Altmeyer et al. assumes some sort of recognition. This recognition is not based on a direct sequence analysis, but only those chain molecules with a proper fold are accepted by the replicase for copying. Such a recognition mechanism can be realized by geometric constraints, or to speak in terms of immunology, by requiring the presence of proper conformational epitopes1. Studies investigating the correlation statistics of folding at the level of secondary structure for RNA [20] have shown that even a single monomer substitution can change the resulting structure considerably. This feature is expected to be preserved in tertiary models of folding. As demonstrated in [3], even comparably low replication fidelities are then sufficient to maintain a replication system because single point mutations are not problematic anymore: They lead to molecules which cannot be copied further. On the other hand, all molecules presenting the correct epitope to the replicase can be copied which opens a path to add novel functionality to the system: since molecules presenting the correct epitope nevertheless may have different catalytic properties. Of course, there is a problem with this way of producing functional diversity: Accumulation of single point mutation will probably not work. There may be various sequences in sequence space presenting the correct epitope, but these sequences cannot 1 A conformational epitope is part of a molecule, in case of chain molecules a sequence of monomers, that attains a specific three-dimensional structure and forms an antigen that can be recognized by a receptor of the immune system. 203 Morphological Control as Guiding Principle in Physiology and Medical Applications necessarily be connected by a series of single point mutations. Other types of variation, such as some form of cross-over have to be assumed. Metaphorically, a copying process that requires the proper epitope to be presented can be understood as a "morphological checksum": Small changes to the sequence of a given molecules change (with high probability) the checksum, but the mapping from molecules to checksums is surjective (many sequences lead to the same checksum). We are aware that the term "checksum" is not completely adequate (at the end, there is no summation involved), but "checkfold" does not seem to be a better choice. In this essay, we will use checksum, implicitly assuming that it is used in a metaphorical manner. In what follows, the morphological checksums are not anymore dependent on sequences, but on the chemical conditions under which synthesis of the checksum molecule takes place. In other words, we will assume a synthesis that is susceptible to changes of the reaction conditions in an "intelligent" way. We relate the idea of a morphological checksum, understood as an instance of morphological control, to the concept of "danger signals." Danger signals are the core of a view on the immune system developed by Matzinger [27]. Matzinger's theory assumes that the immune system does not so much discriminate between the self and the non-self but is also capable of identifying dangerous settings. We emphasize that the concept of danger signals does by no means imply that there is no self/non-self discrimination and of course, the role of the adaptive immune system is not doubted at all. However, danger signals offer new perspectives about the processes that cause the activation of an immune response. One sign of danger in cellular context is certainly the appearance of unusual chemical conditions. Using morphological checksums, we suggest a mechanism (summarized in Figure 2) for the self-surveillance of a cellular system: 1. We first assume that the cellular chemical network comprises some sub-network S that is connected to all substantial chemical functionalities of the cell. 2. S shall be responsible for the production of some molecule M . M is, e.g., a protein, but the type of the molecule is irrelevant for the theoretical considerations presented here, it could also be a complex sugar or something else. 3. The chemical dynamics leading to the synthesis of M shall be susceptible to perturbations in general but robust to changes of the chemical environment as they occur during the course of "normal" cell functioning. The dynamics leading to M is an instance of morphological control in the sense that M is synthesized properly under conditions that can be regarded as normal (including some "normal" perturbations), but is assembled/folded in a wrong manner under unusual and therefore potentially dangerous conditions. 4. To be wrongly assembled/folded shall have a specific consequence for M . here exemplified with a specific assumption about the molecule's geometry, although other mechanisms are conceivable: Assume M to have a part that is globular in its proper state. Furthermore, M shall contain one or several epitopesD , which are not accessible from the outside. Wrong chemical conditions shall lead to a 204 Morphological Control as Guiding Principle in Physiology and Medical Applications Figure 2: Implementation of danger signals. a) We assume a reaction network with a sub-network S (nodes indicated in green) that is connected to the rest of the network. Under conditions that are deemed "normal," this sub-network synthetizes molecules M . In b) such an M is given; a danger epitope D is present but hidden in the interior of M . If, as indicated in c), conditions differ from "normal," the synthesis performed by the sub-network delivers a molecule M⇤ with a danger epitope exposed to the environment. d) Healthy cells (orange) in a healthy environment (green), or better cells with internal reaction conditions that belong to the class "normal," express and present M , indicated by blue bars. e) Cells with a un-normal internal chemistry express M⇤, indicated by a red circle. f) The same applies for cells in an environment that influences the interior of the cell such that M⇤ is synthesized. Note that the exposition of M⇤ does not necessarily mean that the immune system attacks the cell, but that the immune system gets activated. "turning of the inside out," means a molecule M⇤ that presents epitopes D to the environment. 5. We postulate thatM⇤ , e.g. presented on the surface of a cell, initiates an immune response since the danger epitopesD become visible to the immune system. We emphasize that the immune response needs not necessarily to be directed towards the cell presentingD . It just signals the immune system that somewhere close to the D-presenting cell something potentially dangerous is taking place. As a metaphor, regard the D-presenting cell as an alarm system. The ringing bell itself is not the problem, but it may indicate the presence of a burglar. The mechanism we propose offers, at least as a theoretical construct, a couple of interesting features. First, it describes a way of implicit self-surveillance which is based on the implicit dynamics of the synthesis of specific molecules. The mechanism does not detect the presence of a specific danger but the absence of normal conditions. Therefore, 205 Morphological Control as Guiding Principle in Physiology and Medical Applications it is very versatile, without being overly complicated (Comparable to checksums which do not tell you where a problem occurred but indicate with high probability that something went wrong). Secondly, it is an implementation of a danger signal-based mechanism that for example easily explains why certain vaccines (definitely being non-self) do not initiate an immune response unless an adjuvant (a noxious substance) is added. Only if healthy cells experience stress, the immune system starts to look for strange or foreign epitopes. Thirdly, tumors which start to express mutated proteins but are not attacked by the immune system match into the picture as well. Tumor cells still need to upheld normal functions, otherwise they would probably die. These "normal" functions do not trigger the expression of a D . Many more details can be added to the idea of molecular checksums. In our view, most interestingly is an aspect pointed out by Matzinger in [27]: "The first shift [in thinking about the immune system] came from the realization that the immune system may not be the ultimate controller of immunity. Like most immunologists, I had thought that immunity is controlled by the cells of the "adaptive" immune system (lymphocytes) or the more ancient "innate" immune system (such as macrophages, dendritic cells, and the complement system). I now believe that the ultimate power lies with the tissues. When healthy, tissues induce tolerance. When distressed, they stimulate immunity, and (continuing down this path) they may also determine the effector class of a response. Although it has long been thought that the effector class is tailored to the targeted pathogen (e.g., virus or worm), I now think that it is tailored to the tissue in which the response occurs." The mechanism we presented is compatible with Matzinger's comment if we assume that various types of D exist. First, depending on the type of tissue but secondly also depending on the type of danger, different types of immune responses may be triggered. A tissue or danger dependent activation of immunity via the implicit properties of the synthesis of a molecule with various ternary structures is certainly a highly interesting instance of morphological control. If the morphological checksum approach turns out to be relevant in the presented form or some variant thereof, taking, e.g., into account the sophisticated co-stimulation mechanism known to take effect in the immune system, novel therapeutic approaches are conceivable. What needs to be achieved is a perturbation that does not lead to the expression of D in healthy cells but pushes the tumor cells "over the cliff." In that respect, the theory of danger signals is related to hypothesis 2, in which the optimizations in oncology by application of synergistic therapies have been discussed. 206 Morphological Control as Guiding Principle in Physiology and Medical Applications Discussion The dynamical systems perspective on biological systems is by no means new, and neither is the application of concepts from control theory [35]. We emphasize, however, that morphological control has no analogue to the concept of a "reference" which is central in classical control theory. For example, the support systems for patients with movement impairments do not take over the walking, they only facilitate the control that itself is exerted by the patient. A dynamical systems perspective suggests a classification of diseases into two large categories. The first one in which one or several components of a network of interacting entities are malfunctioning and the second one formed by those in which the individual components all work fine but the system as whole is in a wrong basin of attraction. This classification provoked intensive and fruitful discussions with medical practitioners. Of course, such a strict categorization is almost certainly misguided; even if attractor landscapes of dynamical systems play a role, such as in ventricular fibrillation, the switch from the proper into a wrong basin of attraction is often only possible because some control mechanism preventing such a switch is out of order. However, the molecular perspective taken by modern pharmacology, despite its successes, sometimes prevents an assessment from a systemic point of view. Whether biology has to be understood top-down or bottom-up is subject to intensive debate about the philosophical position of systems biology. For an excellent overview that is accessible also to those outside professional philosophy, we recommend [12]. "Programming" or even influencing structures based on morphological control cannot rely on a deterministic approach, at least not in cases relevant in medical applications. Besides the fact that we do not know the actual state of a system in sufficient detail (be it macroscopic or microscopic), we most often have not even complete knowledge about the attractor landscape in which the dynamics of the system takes place. In case of macroscopic systems, a full finite-element simulation may in principle be able to provide this landscape, but in case of cellular systems, we even lack the complete knowledge about the underlying mechanism shaping the attractor landscape. If the concept of morphological control shall take a prominent role in the support of the development of medical therapies, we will need sophisticated statistical procedures that characterize the underlying dynamical system. Evolution plays a twofold role: First, evolutionary approaches are used to determine optimal control schemes. Second, the fact that physiological systems are the product of evolutionary processes gives us some information about what we can expect from their dynamics and how we can interfere with it. We can capitalize on the fact that the attractor landscape of an evolved system is not arbitrary, but has a semantic interpretation. Sometimes, one wonders how the complex and for humans mostly incomprehensible network of molecular interactions can lead to a phenomenology that can be easily interpreted, as for example given in Eqs. 1. It is crucial to note that the mechanisms of evolution consist of several parts (variation, selection, heredity and reproduction): Whereas it is the genotype that is subject to variation, it is the phenotype that is selected. Selection is based on a relation between the phenotype and its environment, therefore selection gives the phenotype a "meaning" with respect to the 207 Morphological Control as Guiding Principle in Physiology and Medical Applications fitness landscape which is imposed in large parts by the external world. In our view, there is as much "sense" on the genotype level as there is on the phenotype level. For a discussion of this point, see [12]. Semantics is (more or less) directly accessible on the phenotype level e.g. to statistical analysis, whereas on the genotype level, measurable relations between the components of the genetic interaction networks have to undergo a non-trivial mapping before they are related to external parameters. Since we do not know all the details of the genotype-phenotype relation, it is crucial to use, and in some cases to develop, statistical tools for analyzing attractor landscapes. We hypothesized about the possibility of what we called molecular checksums. We thereby assumed a complex molecule to be synthesized in a process that is susceptible to the state of the cell. We postulated the expression of danger signals if cells are subject to unusual and potentially dangerous conditions. On a first glance, the proposed mechanism seems to contrast with findings that show cancer to occur much more prevalent in humans than in (wild) animals; for a review of these findings and an interpretation, see [13]. Greaves points out that modern man lives permanently under conditions that differ from those that were dominant during most of human evolution. Therefore, one could argue that since we experience "unusual" conditions in our everyday life, an additional perturbation caused by mutations should trigger danger signals more easily than in wild life. Besides the fact that Greave's approach is much more sophisticated (and subtle) than described here, we respond that we question the assumption that we permanently live under conditions for which we were not prepared by evolution. Humans tend to use modern technology in order to make life more comfortable. This can be understood as living permanently under "unusual" conditions, but we offer a different interpretation. Living under comfortable conditions means permanently living under ideal conditions, in fact reducing perturbations. Regarded from this perspective, danger signals may not be triggered because the effect of dangerous perturbations is not amplified by natural fluctuations. A foundation for the healthy effect of fluctuations may be found in the theory of threshold stochastic resonance [15]. In [17] and [18] the importance of the interplay between morphological and neural control has been emphasized. This connection plays a role in our first hypothesis, but not in hypotheses 2 and 3. In fact, there is an ongoing discussion about the connection of the nervous and the immune system and it is probably fair to say that the jury is still out. A probably somewhat outdated but accessible review is given by Reiche et al. [31]. We emphasize the engineer's point of view. It is at least plausible that there is a connection between the immune system and what is often called lower neural functions, such as transfer of pain signals. The nervous system usually has available information that could be relevant for the formulation of an immune response, such as the localization of an injury or the circumstances under which it happened. In principle, a transfer of this data from the nervous to the immune system could be beneficial, but broader evidence for such a transfer is, to our knowledge, debatable. Using metaphorical language, it is plausible to assume a connection between the immune system that knows what is going on and the nervous system that knows where it is going on (or at least has positional information additional to that already known to the immune system). 208 Morphological Control as Guiding Principle in Physiology and Medical Applications Figure 3: Roadmap for research agenda. Roadmap In what follows, we present a possible research agenda that is organized in a series of integration steps of morphological control and medical sciences, see Figure 3. Step one consists in the characterization of given attractor landscapes and the prediction of system behavior. One can argue that this is what the various "omics"-sciences are doing. Our main focus concerns the semantics of the attractor landscape with respect to its control functionality. The point of view taken by morphological control emphasizes the IT-perspective and is not so much concerned with the details of molecular mechanisms. Morphological control profits and highlights concepts from software engineering, but in systems where the distinction between hardand software is blurred. Methodologically, we search for statistical procedures for the characterization of attractor landscapes. Thereby, we ask whether the fact that we know these attractor landscapes to be the result of an evolutionary process provides us with information that can be exploited in these statistical procedures. Questions relevant in the clinical context are the one about the stability of attractors and about the factors that influence this stability. Especially the latter one could be of relevance for the pharmaceutical industry. As mentioned in the text, applications range from ventricular fibrillation to the study of cell differentiation [34]. In step two, we ask for ways and means that enable to influence the attractor landscape. We illustrated this in hypothesis 2 by trying to optimize synergistic therapies in oncology. The example we have chosen, namely combined hyperthermia-radiotherapy therapies (HT-RT therapies), is motivated by our personal research background. Other choices are certainly possible. In the next steps, the dynamical systems under consideration are reshaped by external means. That first happens on the macroscopic level in step three as exemplified in hypothesis 1. We claim prosthetics and especially support systems for the elderly to be a scientifically and industrially relevant field. Starting with step four, we suggest applying the lessons learnt in step three to microscopic systems, i.e. cells and tissues. Hypothesis three discusses one possible approach. In step five, we suggest to profit from knowledge accumulated in attempts to construct artificial cells and other artifacts belonging to what is called "Living Technology." The ECLT (European 209 Morphological Control as Guiding Principle in Physiology and Medical Applications Centre for Living Technology) is a competence center for European research projects (e.g. "Programmable Artificial Cell Evolution," PACE or "Matrix for Chemical IT," MATCHIT) in the area of combining rational engineering with evolutionary approaches, especially in what has been termed "biochemical ICT" (COBRA). Research in step five will aim at functionalized microscopic systems equipped with some programmability. As mentioned before, the presented simple model of morphological self-surveillance via molecular checksums hardly accounts for the sophistication and subtleties of the human immune response. However, such a simple approach may be a first step towards a protoimmune system in an engineered artifact, and a true realization of living technology. Finally, in step six, potential applications of artifacts implemented in step five will be brought into (or at least towards) clinical practice. Such a research agenda is a truly interdisciplinary endeavor; we claim, however, that morphological control can be used as a guiding principle. We conclude by quoting a footnote in Matzinger [27]: "Like physicists, who deduced the need for a new particle based on the behavior of the system, Bretscher and Cohn, Lafferty and Cunningham postulated cells and/or signals for which, at the time, there was no evidence. Later experiments showed resoundingly that they were correct. In a similar vein, Janeway postulated a new state for a previously known cell, the APC. Up to that time, APCs were thought to be constitutively active, but a seemingly small glitch in the behavior of the system (the need for adjuvant) led him to suggest that they were normally quiescent and needed to be activated. These insights showed that theoretical biology and physics may have more in common than is sometimes thought." Particle physicists postulating new particles basically expect that nature follows their respective theory, or in other terms that they believe that experimental observations can be explained by a theory with an inner logical structure. We claim that in case of biological control systems, the phenomenological top-level of the behavior of a system can be analyzed assuming such an inner logic to be present and to follow structures as one would expect in (evolution-based) information technology. Bibliography [1] Alberts, B. et al. "Molecular Biology of the Cell", fifth rev. ed. Taylor and Francis, ISBN-10: 0815341067, 2007. [2] Ahn, Andrew C., Muneesh Tewari, Chi-Sang Poon, and Russell S. Phillips. "The Limits of Reductionism in Medicine: Could Systems Biology Offer an Alternative?" PLoS Medicine 3(g), 2006: e208, p 710. [3] Altmeyer, S., R. M. Füchslin, and J. S. McCaskill. "Folding stabilizes the evolution of catalysts." Artificial Life 10(1), 2004: 23-38. [4] Auffray, Charles, Zhu Chen, and Leroy Hood. "Systems medicine: the future of medical genomics and healthcare." Genome Medicine 1(1), 2009: 2. 210 Morphological Control as Guiding Principle in Physiology and Medical Applications [5] Boerlijst, M. A., and Paulien Hogeweg. "Spiral wave structure in pre-biotic evolution: Hypercycles stable against parasites." Physica D 48, 1991: 17 28. [6] Collins, J. J., Carson C. Chow, and Thomas T. Imhoff. "Stochastic resonance without tuning." Nature 376, 1995: 236 238. [7] Costa, Jose. "Systems pathology: A critical review." Molecular Oncology 6, 2012: 27 -32. [8] Eigen, Manfred, John S. McCaskill, and Peter Schuster. "The Molecular QuasiSpecies." Adv. Chem. Phys. 75, 1989: 149-263. [9] Füchslin, R. M., S. Altmeyer, and J. S. McCaskill. "Evolutionary stabilization of generous replicases by complex formation." Europ. Phys J. B 38(1), 2004, 38 Ausg.: 103-110. [10] Füchslin, Rudolf M., et al. "Morphological Computation and Morphological Control: Steps Toward a Formal Theory and Applications." Artificial Life 19(1), 2013: 9-34. [11] Füchslin, Rudolf M. , and John S. McCaskill. "Evolutionary self-organization of cell-free genetic coding." Proc. Natl. Acad. Sci. USA 98(16), 2001: 9185-9190. [12] Gatherer, Derek. BMC Systems Biology 4, 2010: 22. [13] Greaves, Mel. "Darwinian medicine: a case for cancer." Nature Reviews Cancer 7, 2007: 213-221. [14] Guirado Llorente, D., et al. "Low dose hypersensitivity in mulicellular tumour spheroids." Radiother. Oncol. 96 suppl. 1, 2010: 607-608. [15] Hänggi, Peter. "Stochastic Resonance in Biology." Chemphyschem. 3(3), 2002: 285290. [16] Harry, J. D., J. B. Niemi, A. A. Priplata, and J. J. Collins. "Balancing Act." IEEE Spectrum 42(4), 2005: 37. [17] Hauser, Helmut, Auke Ijspeert, Rudolf M. Füchslin, Rolf Pfeifer, and Wolfgang Maass. "Towards a theoretical foundation for morphological computation with compliant bodies." Biological Cybenetics 105(5-6), 2011: 355-370. [18] Hauser, Helmut, Auke J. Ijspeert, Rudolf M. Füchslin, Rolf Pfeifer, and Wolfgang Maass. "The Role of Feedback in Morphological Computation with Compliant Bodies." Biological Cybernetics 106, 2012: 1-19. [19] Heinrich, Reinhart, and Tom A. Rapaport. "Generation of nonidentical compartments in vesicular transport systems." J. Cell Biology 168(2), 2005: 271-280. [20] Hofacker, Ivo L., Walter Fontana, Peter F. Stadler, Sebastian Bonhoeffer, Manfred Tacker, and Peter Schuster. "Fast Folding and Comparison of RNA Secondary Structures (The Vienna RNA Package)." Monatsh. Chem. 125, 1994: 167-188. 211 Morphological Control as Guiding Principle in Physiology and Medical Applications [21] Iida, Fumiya, and Cecilia Laschi. "Soft Robotics: Challenges and Perspectives." Procedia Computer Science 7, 2011: 99-102. [22] Johnston, Wendy K., Peter J. Unrau, Michael S. Lawrence, Margaret E. Glasner, and David P. Bartel. "RNA-Catalyzed RNA Polymerization: Accurate and General RNA-Templated Primer Extension." Science 292, 2001: 1319-1325. [23] Knox, Sarah S. "From 'omics' to complex disease: a systems biology approach to gene-environment interactions in cancer." Cancer Cell International 10(11) doi:10.1186/1475-2867-10-11, 2010. [24] Lambin, P., E.P. Malaise, and M.C. Joiner. "Might intrinsic radioresistance of tumour cells be induced by radiation?" Int. J. Radiat. Oncol. Biol. 69, 1996: 279-90. [25] Lipsitz, Lewis A. "Aging as a process of complexity loss." In Complex Systems Science in Biomedicine, von Thomas S. Deisboeck and J. Yasha Kresh, ISBN-10: 0-387-30241-7, 641-654. Springer Inc., 2006. [26] Luchsinger, R. H. , and M. Bräker. "A novel pneumatic actuator with Tensairity." In Design & Nature V, von C.A. (ed.) Brebbia. Southampton: WIT Press, 2010. [27] Matzinger, Polly. "The Danger Model: A Renewed Sense of Self." Science 296, 2002: 301 305. [28] Pfeifer, Rolf, Max Lungarella, and Iida Fumiya. "Self-Organization, Embodiment, and Biologically Inspired Robotics." Science 318, 2007: 1088-1093. [29] Pfeifer, Rolf, and Josh Bongard. How the body shapes the way we think. Brdaford Books, MIT Press, 2007. [30] Priplata, A. A., J. B. Niemi, J. D. Harry, L. A. Lipsitz, and J. J. Collins. "Vibrating insoles and balance control in elderly people." The Lancet 362, 2003: 1123-1124. [31] Reiche, Edna M., Sandra O. Nunes, and Helena K. Morimoto. "Stress, depression, the immune system, and cancer." The Lancet Oncology 5(10), 2004: 617-625. [32] Scheidegger, S., G. Lutters, and S. Bodis. "A LQ-based kinetic model formulation for exploring dynamics of treatment response of tumours in patients." Z. Med. Phys. 21, 2011: 164-173. [33] Scheidegger, Stephan, and Rudolf M. Füchslin. "Kinetic model for dose equivalent – an efficient way to predict systems response of irradiated cells." Proceedings of ASIM 2011, 2011. [34] Vilani, Marco, Alessia Barbieri, and Roberto Serra. "A Dynamical Model of Genetic Networks for Cell Differentiation." PLoS ONE 6(3), 2011: e17703. [35] Wellstead, Peter, Eric Bullinger, Dimitrios Kalamatianos, Oliver Mason, and Mark Verwoerd. "The role of control and system theory in systems biology." Annual Reviews in Control 32, 2008: 33-47. 212 Morphological Control as Guiding Principle in Physiology and Medical Applications [36] Winslow, Raimond L., Natalia Trayanova, Donald Geman, and Michael I. Miller. "Computational Medicine: Translating Models to Clinical Care." Sci. Transl. Med. 4, 2012: 158rv11. 213 Abstract: A theory, by definition, is a generalization of some phenomenon observations, and a principle is a law or a rule that should be followed as a guideline. Their formalization is a creative process, which faces specific and attested steps. The following sections reproduce this logical flow by expressing the principle of Morphological Computation as a timeline: firstly the observations of this phenomenon in Nature has been reported in relation with some recent theories, afterward it has been linked with the current applications in artificial systems and finally the further applications, challenges and objectives will project this principle into future scenarios. The Morphological Computation Principles as a New Paradigm for Robotic Design The Observation of the Morphological Computation Phenomenon in Nature is the First Step for the Formalization of the Principle When we see a dolphin swim, we are marveled by how such a rapid, elegant and strong movement can seem so simple. The dolphin motion has been subject of study since the time of Aristotle. In recent years, it has been found that the high velocities reached by these animals are possible thanks to the formidable mechanical properties of their body. The motion in water is extremely difficult due to the density and viscosity of the medium that imply an exceptional longitudinal force that slows the movement down. The combination of the skin structure, of the appendage shapes, and of the behavioral mechanism during motion allows reducing the drag force, enabling the effective and graceful movement that we all can admire [8]. In Nature, the shape, the geometry, the placement and the compliance properties of the body parts define the perception and the interaction with the environment, thus connecting such kind of features with the expressed behavior, synergistically. All these features can be gathered together in the term morphology. We could say that any transformation of information can be named as computing, and thus, in that sense, Morphological Computation endows all those behaviors where computing is mediated by the mechanical properties of the physical body [23, 22, 21]. There are at least three different cases we can use as reference to describe this transformation: • Shape: the case in which the shapes, as body structure, specifies the behavioral response of the agent. • Arrangement: the case in which the geometrical arrangement of the motors, perceptive and processing units implies specific computational characteristics. • Mechanical properties: the case in which the mechanical properties allow emergent behaviors and highly adaptive interaction with the environment, impossible elsewhere. The first point of view opens the way to all that kind of mechanical structures that facilitate the behavioral expression of a particular agent. A special structure allows many animals to move in the environment with elegant and coordinated gestures: the skeleton. All vertebrates share a similar organization evolved with the precise function to assist movements and in addition to protect internal organs. The term Simplexity has recently been used to explain the solutions that Nature found to simplify the control of complex phenomena [1]. For example, the S-shape characteristic of the animal backbone, and in particular the curvature of the neck, allows the disjunction of the rotations of the head respect to the rest of the body (Figure 1 left). This in turn allows the alignment of the head with the ground in order to stabilize the vision system and thus giving a reference for the control of movements. The complexity added in the morphology of the spine helps to simplify the control. A related example of morphology, but in sensing, is 215 The Morphological Computation Principles as a New Paradigm for Robotic Design Figure 1: Examples of the three cases of Morphological Computation in Nature. Shape, left. The S-shape of the backbone allows the disjunction of the rotations of the head respect to the rest of the body. Arrangement, centre. The geometrical arrangement of the perceptive units in the retina reflects the organization of the processing units into the Superior Colliculus, thus simplifying the processing and the control of eye movements. Mechanical properties, right. The mechanical properties of the muscle-tendon system allow emergent behaviors and highly adaptive interaction with the environment. given by the vestibular system, whose organs are precisely aligned with the Euclidean three-dimensional coordinates, thus facilitating gaze pointing in the Vestibular-Ocular Reflex. The second possible aspect of Morphological Computation can be identified in the geometries of the brain areas, especially in those involved in perception and action. The solutions found by the evolution in these areas reflect the simplification of computation allowed by specific relative positions of the neural networks involved. Also this concept is in line with the definition of the term Simplexity. Actually, this term involves some high-order neural organizations, which simplify the complex nature of the environment comprehension in mammalian brains. However, from the Morphological Computation point of view, the geometry of the neural circuitry in the brain effectively reduces the complexity of computation both in perception and in action. A typical example is the geometry of the Superior Colliculus, a small agglomerate of neurons, which forms a major component of the vertebrate midbrain. The general function of this system is to direct gaze toward specific points in the egocentric space (Figure 1 centre). During the saccadic movement, the Superior Colliculus directly maps the stimulus onto an action response, thanks to the arrangement of neurons and synapses [26]. Hence such kind of complex behaviors, fundamental for the active perception of the world, share the same neural organization. Generally speaking, the way how certain neurons are positioned in the brain delineates the perception and the computation of the response. This knowledge is completely ignored by the most of the current vision systems that passively process the information in the images without considering the geometrical arrangement of the sensing elements and without integrating motion in perception. 216 The Morphological Computation Principles as a New Paradigm for Robotic Design The third case reflects the nature of the materials involved in the interaction with the environment. For example, in humans, this can be seen in the knee or in the elbow structures, which facilitate the compliant response of the limb during motion. Activities like walking, jumping or bringing a glass of water to the mouth are simplified by the elasticity present in the muscle-tendon system. Consequently, the higher cognitive functions of the animal brain, do not have to compute the exact amount of the force response. On the contrary, the brain can perform those complex tasks by controlling just a small number of parameters, such as the stiffness of the muscles (Figure 1 right). Therefore, the mechanical structure yields to a simplification of control and becomes an effective element of the whole computation system. Even the recent state of the art in robotics does not completely take advantage of this kind of phenomenon. Although is not the only possible approach, most current humanoid robots, for example, are able to reproduce the human walking exclusively by computing the exact position of each joint at each time step. This concept, known as Zero-Moment Point (ZMP) and theorized for the first time in 1969 [27], is the most famous and used algorithm used for humanoid walking robots. ZMP is still used since its first practical demonstration in Japan in 1984, at Waseda University, Laboratory of Ichiro Kato, in the first dynamically balanced robot WL-10RD of the robotic family WABOT. It could be said, that, in the stated list, a fourth element is missed: the environment. However, the underline message that we are claiming is that the environment represents a crucial factor for all of the listed features. The environment contains information and the body, through its shape, elements arrangement, and mechanical properties, transforms this information for the agent's outcome. In conclusion, the presented features are more intuitive in animals provided with small computing resources. For example, cockroaches rapidly move even in complex and rugged terrains. The simple brain-like controller provided in the cockroaches is coupled with a compliant leg system. The frequent collisions with the ground and obstacles are damped through the mechanical properties and the dynamical control of the legs [9]. The control of locomotion is not a simple cascade of events, where all the computation is made centrally, but both neural and mechanical systems play a role [7]. The computational burden is distributed also to the mechanical characteristics of its gait, avoiding the full control of the legs parameters from the limited central control system. Current Applications and Technologies Emphasize the Role of Morphological Computation as a Design Principle Morphological Computation can stand as a new approach to robot design. New methods for designing and developing robots, or other computational agents (such as prosthesis or exoskeletons), can exploit the principles of Morphological Computation by essentially transferring (a part of) the computational burden, from the control system to the morphology of the agent. It can be seen as an improvement of the design phase, with possible more complex solutions in the bodyware, for keeping the control to a low, manageable level of computational burden. This will lead to a simplification of control in adaptive 217 The Morphological Computation Principles as a New Paradigm for Robotic Design behaviour, or as an enrichment of behaviour with same control complexity. The extreme essence of Morphological Computation can be described as the simplification of movement control made possible by the presence of a bodyware able to cope with the informational content of the environment. The central processing unit is relieved of unnecessary computation which is indeed distributed toward the mechanical property of the body. The straightforward field of application of the principles of Morphological Computation results to be robotics, and all sectors where movement is involved. In robotics, Morphological Computation can dramatically influence the way robots are designed and controlled, and ultimately their effectiveness. If the common paradigm for robot design is mechatronics, where mechanisms, electronics, control, sensors, and power supply are considered as the main components of the system and designed in an integrated way, Morphological Computation has the potential to establish a new paradigm, where control comes first and the mechanisms and sensors are designed with proper morphology and mechanical characteristics in order to obtain movement with fewer control parameters. Morphological Computation could influence fabrication technologies, as well. The need for specific mechanical properties and morphologies, the use of soft materials, the required integration of components (sensors and actuators, primarily) is pushing forward technologies for building robots and robot components. An example of a fabrication technique, which results suitable for implementing Morphological Computation, is Shape Deposition Manufacturing (SDM). It is a Rapid Prototyping technology in which mechanisms are simultaneously fabricated and assembled as well as integrated with all the necessary remote components. The basic SDM cycle consists of iteratively depositing and shaping (basically machining) layers of part material and positioning the robot parts to be embedded in the subsequent step. These cycles result in three key features: (1) building parts in incremental layers allows a complete access to the internal geometry of any mechanism; (2) this access allows embedding actuators, sensors and other prefabricated functional components inside the structure; (3) by varying the materials used in the deposition process, the material properties of the entire structure itself can be spatially varied allowing the introduction of compliance at specific locations of the body. Locomotion techniques can be seen as one of the main topics where Morphological Computation and SDM can be successfully synergistically used for the design and fabrication of smart robots. An example is given by the cockroach-inspired robot developed by Mark Cutkosky at Stanford [3], where a fast running hexapod robot and its fast adaptation are obtained thanks to the mechanical reflex of its compliant knee joints, in analogy with the animal model. Another relevant example, which can be cited in this field, is the the passive dynamic walking [18]. This phenomenon can be obtained through a simple planar mechanism (the motion is two-dimensional) with two legs demonstrating the capability to walk stably down a slight slope with no other energy source, but gravity and no control. This system acts like two coupled pendula. The stance leg acts like an inverted pendulum, and the swing leg acts like a free pendulum attached to the stance leg at the hip. Given sufficient mass at the hip, the system will have a stable limit cycle, which is a nominal trajectory that repeats itself and will return to this trajectory even if slightly perturbed. An extension of the two-segment passive walker is to include knees, which 218 The Morphological Computation Principles as a New Paradigm for Robotic Design provide natural ground clearance without need for any additional mechanisms. McGeer showed that even with knees, the system has a stable limit cycle [19]. Continuum soft limbs found in Nature demonstrated their capability of dexterous movements and soft and delicate interaction with the environment together with the possibility, when necessary, to change the (structure) stiffness and generate relatively high forces [13]. These peculiar capabilities were found in very simple animals, in evolutionary terms, suggesting the existence of an effective way of reducing the control efforts of such limbs. In the octopus, for example, the control of the 8 soft continuum arms with virtually infinite degrees of freedom is handled by a hierarchical (system) organization, where tissues, mechanical properties and their arrangement are of fundamental importance to achieve such performances. The density of the tissues, their packaging in a very particular manner ("muscular hydrostat") and the conical shape of the arms allow a synergetic exploitation of the environment characteristics and together with the use of control primitives allow the implementation of complex movement setting a very few parameters at central level [11]. The presence of all these characteristics and their perfect integration seem to play a fundamental role in the motor behavior and for this reason the octopus-like robot described in [2, 4, 15] took into account their function as well as their reciprocal interaction from a very early stage of design. Only soft or flexible materials have been used in order to leave to the robot the capability of adapting to the environment [14], in particular, when high deformability and squeezing are necessary requirements for the task to accomplish. But the soft nature of the robot is counterbalanced by a selective stiffening system, which allows the arms to actively change their mechanical properties and to exert forces on the environment. This is made possible by an actuation system based on Shape Memory Alloy springs and motor driven tendons arranged mimicking the octopus muscular system (muscular hydrostat), which is the key factor to transform deformation into motion, without the use of rigid supports (Figure 2). Such a system has been successfully used to implement motor control primitives (such as the ones found in the real octopus), which, together with the geometrical shape of the arm, demonstrated the possibility to perform effective and energetically efficient movements, with a very low computational burden. This is a clear example of exploitation of Morphological Computation principles, where mechanical properties of the materials, the arrangement of the active elements (actuators) and the geometrical shapes are used to simplify the implementation of behaviors that otherwise would require a complex control system. As described before, one of the main features of Morphological Computation is given by the arrangement of the system elements both in terms of sensors and actuators. Neuromorphic engineering is an emergent field, which focuses on the design and development of new generation of compact chips able to emulate the neural organization and function of thousands of neurons in electronic devices [20]. Silicon neurons (SiNs), hybrid analog/digital very large scale integration (VLSI) circuits, emulate in hardware the electrophysiological behaviour of real neurons and conductances. Neuromorphic SiN networks are much more efficient than simulations executed on general purpose computers and the speed of the network is independent of the number of neurons or of their coupling [24, 25]. Furthermore, spiking neural processing modules, distributed across multiple 219 The Morphological Computation Principles as a New Paradigm for Robotic Design Figure 2: OCTOPUS robot arm. The soft arm is able to mimic the real octopus arm capabilities by capitalizing on Morphological Computation and Soft Robotics technologies (credit: Massimo Brega The Lighthouse). neuromorphic chips can be interconnected in a manner which is inspired by the nervous system [12]. This approach has been adopted for designing radically different spike-based vision and auditory sensors (e.g., silicon retinas, and silicon cochleas) [6, 16, 17]. Rather than capturing sensory signals in a sequence of static frames, these new bio-inspired devices produce real-time asynchronous spikes from the pixels or sensing elements that receive inputs, in the moment in which they are activated. As in biological sensors, the outputs of these devices are quite sparse, but with very high temporal resolution. The coding scheme used by these devices reduces redundancy and, as a consequence, minimizes computational requirements and power-dissipation figures. Specifically, the cost of processing the sparse output of neuromorphic sensors can be reduced by more than 2 orders of magnitude compared to the cost of processing the outputs of standard sensors [6]. Finally, mechanisms where the system has fewer inputs than degrees of freedom can be cases which exploits the principle of Morphological Computation approach, since the structure is designed to cope with a number of different interactions with the environment guaranteeing the success of the task accomplishment. Underactuated grippers are capable to conform to a wide variety of objects softly and gently, and to hold them with uniform pressure with a very simple control structure [10]. The arrangement of the tendons and the shape of the underactuated device (and partially its material properties) can be varied to increase the grasping flexibility and adaptability and at the same time to reduce the control complexity. 220 The Morphological Computation Principles as a New Paradigm for Robotic Design Next Horizons can be Deduced from Specific and Concrete Applications The application of Morphological Computation principles in robot design can give rise to a new generation of robots with enhanced adaptability and limited number of required control parameters. They will be better suited for real-world applications and in this sense Morphological Computation will contribute to the progress of the broad field of service robotics. This is particularly important for unstructured environments when the external conditions are unknown and may present unpredictable obstacles becoming dangerous or inaccessible for human beings. This kind of scenario represents a challenging task for current robots, which usually show insufficient capabilities of adaptability. With respect to the classical robotics approach, Morphological Computation is able to cope with uncertainty exploiting the body characteristics to adapt to the environment and in some cases to exploit it providing an even richer repertoire of behaviors maintaining the same complexity in the control system. Among the many hostile environments for humans, the underwater environment is one of the hardest to face. Here robots are being used for years, still in form of vehicles, in some cases with robotic manipulators, but with very limited usability where high dexterity should be shown together with soft interaction. Underwater manipulation tasks executed by traditional robot are made ineffective by the necessity of a very highly precise control, often not possible in such conditions. Thus, in many cases, the low capability of adaptability is compensated with a limited level of interaction. However, in some underwater tasks, the physical contact and the interaction with man-made structures or the sea bottom are mandatory: exploration or rescue in wrecks, maintenance of pipelines or other underwater structures, exploration of the sea bottom or reefs. In these cases, Morphological Computation principles could considerably increase the capability of dexterity maintaining the complexity of the control system at a manageable level. Another harsh environment where robots play an essential role is space. Vacuum, extreme temperatures, radiations and different levels of gravity are some of the characteristics of this uncomfortable scenario. When the car-sized robotic rover Curiosity reached the Mars ground, one of the main challenges was to ensure the communication with its operative system from the Earth. Designing a robotic system able to autonomous interact with the environment following the Morphological Computation principle, while the human operator simply controls and decides some high level actions, would unload the data stream from the operator and the robot, reducing communication issues. In biomedical applications, the use of Morphological Computation principles may result at least controversial, but it could lead to important improvements. In a broad field where robots are already used for surgery, endoscopy, rehabilitation and assistance, it is widely reasonable that the human operator, the surgeon or the nurse, should keep the finest and greatest control on every part of the tools used inside the human body. Thus, in such kind of applications, a robot which autonomously interacts with the internal tissues could be even dangerous. However, Morphological Computation principles shed new light in a broad field of technologies, which can be endowed in the biomedical field. To 221 The Morphological Computation Principles as a New Paradigm for Robotic Design clarify this point, let us imagine a possible scenario with a surgeon executing a laparoscopic procedure 1: the most suitable device should be able to safely interact with the environment letting the doctor concentrate only on the operation site. It should present a soft end-effector able to actively vary its stiffness in selected parts, while the support structure should actively interact with the body environment modifying its volume to adapt itself to the tissue walls [5]. In industrial robotics, as well, we may envisage an application of Morphological Computation principles. In many industrial productions there is still a part of the processes that needs to be performed by human operators. This is mainly due to the higher dexterity of human beings in those tasks where the object of the manipulation can change in shape or position and require some levels of adaptability. In some cases, this gives rise to mistreatment of human resources. Reaching human-like level of dexterity would allow a proper use of machines. The Main Challenges and Ambitious Objectives Shed Light on the Future Scenarios The concrete implementation of Morphological Computation principles in robots is still a creative process, widely left to the understanding and personal perception of the designer. More structured design guidelines or methods would probably help the design process and the development of robots, which incorporate Morphological Computation principles. The same stands for some fundamental enabling technologies related to materials and fabrication techniques: the need for specific mechanical properties and morphologies, the use of soft materials, the required integration of components (sensors and actuators, primarily) are pushing forward the technologies for building robots and robot components, which would bring a strong boost in the capability of exploiting the power of the Morphological Computation approach. New materials combining different properties (mechanical, but also electrical, chemical and thermal), composite materials and compounds would be the basic bricks to be combined with fabrication technologies like (but not limited to) SDM enabling the production of a wide range of possible new structures, mechanisms and systems. Despite the described advantages coming from the shift of the adaptability control responsibility from the central processing unit to the mechanical characteristics of the peripheries, an evident limit arises when the agent has to tackle problems in which a fine control of the environment interaction is needed. As a child gradually learns to execute smooth and refined movements in controlling his own limbs, only applying the principles of Morphological Computation could result limiting and thus the design process should take into account the possibility to include specific learning mechanisms for precise and accurate, fully-controlled, movements. In animal brains two subcortical structures solve the problem of motor control. The basal ganglia2 are specialized for learning from the 1 A surgical procedure in which a fibre-optic instrument is inserted through the abdominal wall to view the organs in the abdomen or permit small-scale surgery, see also Laparoscopic surgery. 2 The basal ganglia (or basal nuclei) comprise multiple subcortical nuclei, of varied origin, in the brains 222 The Morphological Computation Principles as a New Paradigm for Robotic Design reward/punishment signals coming after a specific action, thus formalizing an action selection mechanism which enables the organism to adapt to the different circumstances. The cerebellum3 is instead specialized for learning from errors between sensory outcomes associated with motor actions and the relative expectations for these sensory outcomes associated with those motor actions. Therefore, the basal ganglia select one of the possible actions to perform, while the cerebellum refines the implementation of a given motor plan, to make it more accurate, efficient and well-coordinated. The presence of these neural structures in almost all the animal brains provides an elegant parallelism with the kind of challenges that robotics has to explore. As a consequence, in the Morphological Computation framework, the mechanical structure could likely be complex, but different strategies of motor control should co-exist at the same time, showing adaptation capability to uncertainty and learning behavior to specific movements across a wide range of motor tasks. A final consideration concerns the interaction of humans and machines. Recent progresses of robotics are providing sophisticated systems which can perform complex tasks in the service of humans. If new robotics technologies allow to build robots with more degrees of freedom and improved dexterity, on the other hand the human users are facing an increasing complexity in operating high-tech robots. The reduction of the number of control parameters, without reducing the number of degrees of freedom and dexterity becomes an ideal solution in this perspective making Morphological Computation a fundamental ally in the robot design process. Bibliography [1] A. Berthoz and G. Weiss. Simplexity: Simplifying Principles for a Complex World. Yale University Press, 2012. [2] M. Calisti, M. Giorelli, G. Levy, B. Mazzolai, B. Hochner, C. Laschi, and P. Dario. An octopus-bioinspired solution to movement and manipulation for soft robots. Bioinspiration & Biomimetics, 6(3):036002, September 2011. [3] G. J. Cham, A. S. Bailey, E. J. Clark, J. R. Full, and R. M. Cutkosky. Fast and Robust: Hexapedal Robots via Shape Deposition Manufacturing. International Journal of Robotics Research, November 2002. [4] M. Cianchetti, A. Arienti, M. Follador, B. Mazzolai, P. Dario, and C. Laschi. Design concept and validation of a robotic arm inspired by the octopus. Materials Science and Engineering: C, 31(6):1230–1239, August 2011. of vertebrates, which are situated at the base of the forebrain. The basal ganglia are associated with a variety of functions including: control of voluntary motor movements, procedural learning, routine behaviors or "habits" such as bruxism, eye movements, cognition and emotion. See also basal ganglia. 3 The cerebellum (Latin for "little brain") is a region of the brain that plays an important role in motor control. It may also be involved in some cognitive functions such as attention and language, and in regulating fear and pleasure responses; its movement-related functions are the most solidly established. See also Cerebellum. 223 The Morphological Computation Principles as a New Paradigm for Robotic Design [5] M. Cianchetti, T. Ranzani, G. Gerboni, T. Nanayakkara, K. Althoefer, P. Dasgupta, and A. Menciassi. Soft robotics technologies to address shortcomings in today's minimally invasive surgery: the STIFF-FLOP approach. Soft Robotics, 1(2):122– 131, 2014. [6] T. Delbruck and B. Linares-Barranco. Activity-Driven, Event-Based Vision Sensors. IEEE International Symposium on Circuits and Systems, January 2010. [7] M. H. Dickinson, C. T. Farley, R. J. Full, M. A. Koehl, R. Kram, and S. Lehman. How animals move: an integrative view. Science, 288(5463):100–106, April 2000. [8] F. E. Fish. The myth and reality of Gray's paradox: implication of dolphin drag reduction for technology. Bioinspiration & Biomimetics, 1(2):R17–R25, May 2006. [9] R. J. Full and D. E. Koditschek. Templates and anchors: neuromechanical hypotheses of legged locomotion on land. Journal of Experimental Biology, 202(23): 3325–3332, 1999. [10] S. Hirose and Y. Umetani. The development of soft gripper for the versatile robot hand. Mechanism and Machine Theory, 13(3):351–359, January 1978. [11] B. Hochner. An embodied view of octopus neurobiology. Current biology, 22(20): R887–92, October 2012. [12] G. Indiveri, B. Linares-Barranco, T. J. Hamilton, A. van Schaik, R. EtienneCummings, T. Delbruck, S. Liu, P. Dudek, P. Hafliger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, T. SerranoGotarredona, J. Wijekoon, Y. Wang, and K. Boahen. Neuromorphic Silicon Neuron Circuits. Frontiers in Neuroscience, 5, 2011. [13] S. Kim, C. Laschi, and B. Trimmer. Soft robotics: a bioinspired evolution in robotics. Trends in Biotechnology, 31(5):287–294, April 2013. [14] C. Laschi and M. Cianchetti. Soft Robotics: new perspectives for robot bodyware and control. Frontiers in Bioengineering and Biotechnology, 2(3), 2014. [15] C. Laschi, M. Cianchetti, B. Mazzolai, L. Margheri, M. Follador, and P. Dario. Soft Robot Arm Inspired by the Octopus. Advanced Robotics, 26(7):709–727, January 2012. [16] S. Liu and T. Delbruck. Neuromorphic sensory systems. Current Opinion in Neurobiology, January 2010. [17] M. Mahowald. An Analog VSLI System for Stereoscopic Vision. Kluwer Academic, 1994. [18] T. McGeer. Passive Dynamic Walking. The International Journal of Robotics Research, 9(2):62–82, April 1990. 224 The Morphological Computation Principles as a New Paradigm for Robotic Design [19] T. McGeer. Passive walking with knees. In Robotics and Automation, pages 1640– 1645. IEEE, 1990. [20] C. Mead. Analog VLSI and neural systems. Reading: Addison-Wesley, 1989. [21] R. Pfeifer and F. Iida. Morphological computation–connecting brain, body, and environment. Creating Brain-Like Intelligence, pages 66–83, 2009. [22] R. Pfeifer, F. Iida, and G. Gómez. Morphological computation for adaptive behavior and cognition. International Congress Series, 1291:22–29, June 2006. [23] R. Pfeifer, J. Bongard, and S. Grand. How the Body Shapes the Way We Think: A New View of Intelligence. A Bradford Book, 2007. [24] J. Schemmel, J. Fieres, and K. Meier. Wafer-scale integration of analog neural networks. Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence), pages 431–438, 2008. [25] R. Silver, K. Boahen, S. Grillner, N. Kopell, and K. L. Olsen. Neurotech for neuroscience: unifying concepts, organizing principles, and emerging tools. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience, 27(44):11807– 11819, October 2007. [26] N. Tabereau, D. Bennequin, J. Slotine, A. Berthoz, and B. Girard. Geometry of the superior colliculus mapping and efficient oculomotor computation. Biological Cybernetics, January 2007. [27] M. Vukobratović and D. Juricić. Contribution to the synthesis of biped gait. IEEE Transactions on Biomedical Engineering, 16(1):1–6, January 1969. 225 Abstract: Recently, two theoretical models for morphological computation have been proposed [13,14]. Based on a rigorous mathematical framework and simulations it has been demonstrated that compliant, complex physical bodies can be effectively employed as computational resources. Even more recent work showed that these models are not only of theoretical nature, but are also applicable to a number of different soft robotic platforms. Motivated by these encouraging results we discuss a number of remarkable implications when real physical bodies are employed as computational resources. Morphological Computation – The Physical Body as a Computational Resource Introduction The two underlying theories of [13] and [14] are based on two very different theoretical bases. The first one [13] employs theoretical results by Boyd and Chua [4] from signal processing and discusses mathematical operators as a form to describe computation. Such operators are mathematical objects that can, for example, map (input) functions onto (output) functions.1 In the context of robots we talk about functions in time, for example, the mapping from sensory input streams onto sensory output streams. The second model introduced by Hauser et al. [14] is based on a tool from nonlinear control theory (i.e., feedback linearization, see [17] for more details) and, consequently, uses nonlinear dynamical systems to describe computation. Note that smooth non-autonomous dynamical systems, wherein the input function is interpreted as a nonautonomous driving term, are mathematical operators as well, since they map input functions in time onto output functions in time. Interestingly, despite the obvious differences in their choices of mathematical tools for the two theories, the two derived corresponding computational setups look quite similar. Figure 1a shows the considered setup for the second theoretical model, while the setup based on the first theoretical model is very similar, it simply lacks the feedback loop. The reason is that both mathematical models are motivated by the same machine learning technique called reservoir computing (RC). This is a supervised learning approach, which has been quite successful in a number of challenging tasks including nonlinear mapping of input streams onto output streams (i.e., learning to emulate complex, nonlinear differential equations). A good overview of its success story can be found in [27]. At the core of RC lays the so-called reservoir, a randomly initialized high-dimensional, nonlinear dynamical system, which maps the typically low-dimensional input (stream) onto its high-dimensional state space in a nonlinear fashion. In that sense the reservoir takes the role of a kernel (in the machine learning sense, i.e., the nonlinear projection of a low dimensional input into a high-dimensional space, see, e.g., [13] for a discussion). In addition, the reservoir, being a dynamical system, has the inherent property to integrate input information over time, which is obviously beneficial for any computation that needs information on the history of its input values. It is important to note that the reservoir is not altered during the learning process. Although it is randomly initialized, its dynamic parameters are fixed afterwards. In order to learn to emulate a desired input output behavior (to be more precise, a desired mapping from input streams to output streams), one has to add a linear output layer, which simply calculates a linearly weighted sum of the signals of the high-dimensional state space of the reservoir. These output weights are the only parameters that are adapted during the learning process. Figure 1b shows the classical RC setup, where the nodes represent simple, but nonlinear differential equations. They are randomly connected with each other and, therefore, build a complex, nonlinear dynamic system, i.e., the reservoir. 1In more general terms an operator is defined as the mapping from a vector space or module onto another vector space or module. Since functions can be understood as elements of a function space, which under the usual operation of addition and scalar multiplication form a vector space, the language and concepts of operator theory are applicable for the study of mappings from functions to functions. 227 Morphological Computation – The Physical Body as a Computational Resource (a) Setup as used in the simulations in [13] and [14] (b) Classical RC as described in [27] (c) Schematics of the setup of the octopus experiments described in [24, 23, 22] (d) Schematic setup of the quadruped robot with soft spine from [34] Figure 1: Overview of different RC setups referred to in the text. (a) is the classical RC setup with generic nodes. (b-d) are schematics of implementations of Physical RC where real physical bodies are employed as reservoirs and, therefore, as a computational resources. 228 Morphological Computation – The Physical Body as a Computational Resource The remarkable conclusion is that we can learn to emulate complex, nonlinear computations (e.g., complex operators like nonlinear dynamical systems representing, e.g., a nonlinear controller) by simply finding some linear and static output weights. Hence, the task to learn to emulate a nonlinear dynamical system is, with the help of the reservoir, reduced to simple linear regression. Another remarkable property of the RC approach is that the reservoir is not bound to be chosen from a specific class of dynamical systems. It only has to exhibit a number of rather generic properties to be useful as a reservoir: It should be high-dimensional, nonlinear and exhibit stable temporal integration.2 As a result there exist different flavors of RC, e.g., echo state networks [18] or liquid state machines [20], employing different types of artificial reservoirs. For more details we refer to [27]. In the more recent years the RC approach has outgrown its pure machine learning domain. Looking at the desired properties for the reservoir (i.e., it should be a complex, nonlinear dynamical system with time integration capabilities) one can see that these properties cannot only be found in artificial dynamical systems that have been designed explicitly with this purpose in mind. Actually, a number of real physical systems exhibit these features as well and, hence, can potentially serve as reservoirs. One of the first examples along this line of thought was the "water in the bucket" experiment of [8], where the dynamic, nonlinear effects of a water surface have been successfully exploited as a reservoir to carry out vowel classification. However, recently a whole series of applications, especially, in nonlinear optics and soft robotics have extended this notion quite impressively, even so far, as there has been coined the term Physical RC to describe this research approach. In this work we concentrate on a discussion on Physical RC in the context of robotics. For more details on applications in nonlinear optics, we refer to [1] or [28, 7]. The theoretical setups proposed in [13] and [14] fall also into this category. They provide the theoretical basis for a wide range of real physical systems that can serve as reservoirs. Even the presented (rather abstract) networks of nonlinear springs and masses (and their simulations) are meant to provide a generic description of real physical bodies capable of serving as computational resources (compare Figure 1a). A similar line of thought has been applied by Caluwaerts et al. [5] by simulating a tensegrity structure and use it as a physical reservoir. There exist also a number of simulations employing more concrete platforms. For example, Nakajima et al. [23] implemented a sophisticated bio-inspired simulation of an octopus arm and demonstrated its computational power. Sumioka et al. [29] simulated a rather simple model of a human musculoskeletal system and employed it successfully as a reservoir. Hauser and Griesbacher [12] used simulations to show that a compliant physical body can be used to control a rigid robot arm. Bernhardsgrütter et al. [2] extended this work by letting the soft body structure grow based on rules encoded as L-systems. Next to the simulation results, which are already remarkable, a series of work has 2In RC literature temporal integration is referred to as the fading memory property. In the context of dynamical systems this means we need to have an exponentially stable system that has one point of equilibrium or stays "close enough" to such a point for a proof see [3]. 229 Morphological Computation – The Physical Body as a Computational Resource demonstrated the applicability to real physical platforms. For example, a soft silicone based octopus arm has been used by Nakajima et al. to carry out computations and to implement a feedback controller [22]. Zhao et al. [34] constructed a quadruped platform that employs a bio-inspired soft spine as a reservoir to control the locomotion of the robot. Even tensegrity structures have been built and employed in such a Physical RC approach with the goal to use them for exo-planetary exploration [6]. While one might understand Physical RC as a simple extension of the original RC approach, we strongly believe that there are a range of remarkable implications, especially, when you consider implementations in the context of morphological computation on real-world robotic platforms like soft robots. Suddenly, abstract terms and notions from machine learning have real-world, physical meanings. We believe, as a consequence, this initiates a radical change of viewpoint, which implies a paradigm shift with respect to robotic design and even to computational theory. In the following sections we are going to highlight and discuss some of these implications and we hope to inspire researchers to enter this young and exciting field of research. The Body is not a Computer, but . . . The original RC approach is a pure machine learning technique. Its main purpose is to do computation, or to be more precise, to provide a learning setup to emulate a desired computation. However, if you work with a real physical body as a reservoir you suddenly have to deal with a completely different point of view and, as a result, abstract terms (like fading memory, kernel property, etc.) have real world implications and interpretations. Typically, the body or (body parts) of biological systems and of robots have (a set of) specific functions and are not explicitly built for computation. For example, the leg is used for locomotion or to kick a ball. The hand is meant to grasp an object or to play piano. The Physical RC approach, however, implies that on top of these functions we can exploit the dynamical properties of these body parts to carry out relevant computational tasks (e.g., to provide an appropriate continuous control signal based on the continuous sensory streams to stabilize the movement when kicking the ball). One could even argue that we get this computation for free. As one can see in Figures 1(b-d) we typically consider input(s) to our physical body in form of forces or torques. Either the environment (or some other agent) applies them or they come directly from an active degree of freedom, i.e., any type of actuation, within the robot. The body does not "know" that it is part of a computational device. It simply obeys the laws of physics and it reacts accordingly. Note that this also implies that the body does not overnor under-compensate, since it is a stable physical system.3 The proposed setup simply adds some linear readouts to the body to complete the computation. The body would react exactly the same, if there were no readouts at all. If this output is used, e.g., in a feedback loop as a control signal for the robot, the behavior of the robot of course should be different if we close the loop by adding the readout. However, the 3For example, a linear mass-spring-damper system reacts with a force proportional to the perturbation not more, nor less. 230 Morphological Computation – The Physical Body as a Computational Resource body still does not "know" that it is part of this computational loop. This raises the philosophical question, sometimes referred to as the teleological principle, whether the body is carrying out computation at all. Although, we are not trying to answer this question here,4 we can argue that Physical RC can still be of practical value, if it is able to provide a beneficial mapping of input streams onto output streams with the help of the physical body. Another interesting implication of this property of "ignorance" of the body (i.e., it behaves the same, whether we add readouts or not) is that we can employ the same body for multiple computations on the same input at the same time. We simply have to add the according number of readouts. We refer to that as multitasking and this has previously been demonstrated in simulations in [13] and [14] as well on real platforms, see [22]. The Power of Linear Regression As pointed out in the introduction the RC setup needs on one hand a reservoir, which is basically a complex nonlinear, dynamical system with fixed parameters for the dynamics, and on the other hand a linear, static readout, which is adapted during the learning procedure. As a consequence, the task to learn to emulate complex, nonlinear operators/dynamical systems can be, with the help of the reservoir, reduced to the much simpler task to find some linear output weights, i.e., to carry out linear regression. It has been argued in [13] and [14] that if learning was successful, all the nonlinearity and memory that is needed for the emulation of the given computational task can be considered to be outsourced to the physical body in such a setup5. As also discussed in [13] this fact implies a number of remarkable properties. First of all, linear regression is fast and it is even guaranteed to find a global optimum. Furthermore, linear regression "picks" (i.e., assigns bigger weights) to signals that are more relevant to produce the desired output, i.e., it naturally deprecates irrelevant information. Linear regression also provides the possibility to employ online learning, e.g., Recursive Least Square (RLS) and others. Caluwaerts et al. demonstrated this possibility in their simulation work with tensegrity robots, see [5], where the structure learned online to improve its locomotion. Besides these nice properties linear regression has also a more hidden advantage. It averages over conflicting information. This feature is especially important when a feedback setup (as presented in [14] see also Figure 1a) is used. In general, such a feedback setup is needed to emulate more complex computations, like stable nonlinear limit cycle (e.g., for locomotion) or dynamical systems with bifurcation behavior or with multiple equilibrium points (i.e., analog finite state switching machines). The process to learn in such a case takes place in open loop, i.e., the optimal feedback (the target output) is fed back to the body and the internal signals 4For an excellent discussion of this question we refer to the submission of Hoffmann and Müller in this e-book. 5The argument is that the readout is only linear and does not have any notion of memory (i.e, it is static). Hence, in such a setup nonlinearity and memory can only be contributed by the physical body. 231 Morphological Computation – The Physical Body as a Computational Resource (sensory information) are collected to calculate the optimal weights. In simulations, if we only use this data, we are able to learn the (almost) perfect trajectory of a given target limit cycle. However, as soon as we close the loop already numerical imprecisions drive the system away from this trajectory. It is not stable. The system is not robust. However, if we superimpose noise to the learning data, linear regression averages over a region in the state space (i.e., some space around the nominal trajectory) and inherently learns a region of attraction, which results in a stable limit cycle. The conclusion is that noise is crucial for the robustness in a closed loop setup. Remarkably, for example, in the octopus arm setup of [22] the necessary noise to learn a robust limit cycle is provided by the physical environment (e.g., from the sensors or any other noise of the physical body and the environment). Another aspect of robustness can be observed when we use linear regression in its online version. If, for example, a sensor is broken (e.g., producing only a zero value or only noise), its value will be simply deprecated over time by the online learning approach, i.e., it will receive a decreasing weight. Furthermore, we speculate that even in the case when any other part of the body is broken (e.g., a spring, a motor) the setup is potentially highly resilient as well. Note that in this context one could use also more sophisticated online learning approaches to take advantage of the Physical RC setup. For example, one could employ reinforcement learning techniques (see [30] for an overview) that try to improve a reasonable working trajectory based on a given performance measurement (e.g., speed in locomotion). Another possibility is to combine it with learning techniques that are information theory driven, as for example used in [21]. Such approaches are especially useful if we don't have a desired target function and the system has to learn to exploit its "eigen-dynamics." Finally, we want to point out that linear regression is also inherently unbiased with respect to what kind of sensor types (e.g., gyroscope, pressure, force, visual, etc.) are used. It is also able to deal with different scales, changing the number of sensors being used, or redundant information.6 Interestingly, biological systems exhibit a huge number of internal sensors and, hopefully, we will see a growth in the number of sensors in robots in the near future as well. The Limitations of the Physical World A big difference between the classical RC approach and the use of real physical bodies are mechanical constraints. While in the machine learning setup you can virtually choose any parameters values to fit your task, when you use a real physical platform you have to deal with given limitations. The first one is the constraint related to frequency. A (stable) physical system works as a (nonlinear) low-pass filter. Mechanical parts cannot follow arbitrary frequencies and might be able to respond only at lower frequencies. The second constraint (when compared to the virtual RC approach) is the transfer and distribution of information. In the standard RC approach, typically, all nodes are 6Clearly, if all signals are linearly dependent, we don't get the necessary computational power. 232 Morphological Computation – The Physical Body as a Computational Resource allowed to be connected with any other node.7 However, in a real physical systems the information has to travel through the body by mechanical interaction. For example, if the octopus robot arm of Figure 1c is excited at the shoulder (on top in the depicted platform) than it will take some time until this information is carried along the passive silicone structure to have some effect on the tip of the arm. Although this seems to look like a disadvantage, after all we discuss here constraints, this can also be the right amount of fading memory that is needed for this robotic structure to be useful in certain computational tasks. This points to the fact that it makes sense to consider the computational role of physical bodies already during the design process. The third constraint we discuss is noise. This could be introduced by physical effects (e.g., sensory noise) or by numerical imprecision, when the involved signals are digitalized. While in general noise is perceived as a disadvantage, in the case of the RC setup for morphological computation, as already pointed out, it can be an advantage. In the case of a learning setup, where we want to use the body in a closed loop (Figure 1a), noise makes sure to provide a broad enough set of learning data points around the desired trajectory to get a robust closed loop system. Of course the amplitude of the noise is crucial. Remarkably, in our experiments with real physical platforms we have not encountered any problems with respect to that so far. It seems the existing noise in the system, e.g., from the sensor and/or the body itself or the environment, provides the right amount and type of noise. In the context of noise we would also like to point the interested reader to the work of Hänggi [11], where the author demonstrated the usefulness of noise in biological systems in other context. In summary, it is important to consider physical constraints present in robotic platforms. To be more specific, we argue that is important to consider them already during the design process as they directly influence the computational power of the physical body. Although, constraints constitute a problem and they clearly point to limitations of morphological computation, they can also help us to decide which part should be or can be outsourced to the physical layer and which part should be retained in the more classical electronic controlling units. The "Body" and the "Brain" In this section, we follow up on the question what should be or what can be outsourced to the morphology of the robot and which part should be implemented in the abstract, digital layer of the CPUs. Where should we draw the line between morphological computation and pure digital computation? Before we do that we have to clarify what we mean by morphology (physical body) and digital layer (controller) in this context. Clearly, any virtual controller or digital controlling unit needs a physical embodiment. This could be, for example, a simple CPU, a complex signal processor, or even a full-blown computer. However, when we 7The Liquid State Machine approach [20] limits that by allowing some Gaussian probability for the connections, i.e., nodes are more probable to be connected to closer nodes than to ones that are farther away. This is done in order to mimic the connectivity pattern in the brain. 233 Morphological Computation – The Physical Body as a Computational Resource talk about morphology we are explicitly not considering classical computational architectures. The reason is that in these cases the physical implementation is designed to be as independent from the physical realization (i.e., the morphology) as possible. This is exactly the opposite of what we consider to be useful in morphological computation. A computer architecture is meant to be robust against any external influences, which are not defined as computational (digital) input. Loosely speaking, we don't want any bit to flip because an agent applies forces to the physical body of the computer, nor should any environmental change (like increasing temperature) influence the outcome of the computation.8 On the other hand, in the morphological computation setup input is received directly through the body by interaction with the environment including other agents in form of forces. The robot body serves as actuator (output it applies forces) as well as sensor (react physically to input forces). Another difference is that classical computational hardware is based on digital information (because it makes it more robust against external influences), while morphological computation lives in the continuous realm – it is analog computation. For a further and more in-depth discussion we refer to [10]. Coming back to the notion of morphology, when we talk about a physical body or morphology we refer to the part that is involved in morphological computation. When we use the term controller, or virtual or abstract computation we refer to the part that is implemented in form of classical computational architectures (e.g., a software program in a CPU). Note that the abstract (second) layer (in macroscopic biological systems these are neural networks) is typically on top of the morphological computation layer. Another important point is that classical robot designs (built out of rigid body parts connected with high torque servos) don't have any deliberate computation in the physical layer, hence, any computation in the robot is carried out only in the abstract controller layer. Having clarified the terms we can now assess what kind of computations should be outsourced to the morphological layer and which computation should remain in the domain of the abstract control level. Based on the discussion of the previous section on constraints we can argue that it makes sense to outsource only computations that are in the frequency range inherent to the physical body. This would allow us to take advantage of this analog domain, which does not suffer from errors and delays introduced by digitalization processes. Hence, such computation is extremely fast. This is especially of advantage in computations that don't need to consider the long time history of inputs nor the integration of multiple sensory information. Reflexes are an excellent example for potential applications. Another typical field of application of morphological computation is given in tasks, where physical interaction with the environment or another agent is predominant. There is no need to artificially transfer the sensory information to a central processor, as it can be "used" directly and locally in a morphological computation setup. For example, legged locomotion is based on the appropriate interaction of the legs with the ground. But also efficient flying and swimming depend on the "appropriate" physical interaction with the engulfing medium. Grasping is another example where 8The only possibility is if we define a corresponding digital input to the computer as, for example, a thermal sensor. 234 Morphological Computation – The Physical Body as a Computational Resource morphological computation seems to be especially suited. On the other hand, considering the given limitations of the physicality we can also conclude that more complex computational tasks, like planning or complex decision making, should be realized in the digital domain, as we can take advantage of features of classical computer architecture like long-term memory, look-up tables, and others. Of course, there is still a gray area, i.e., computational tasks that can either be implemented in the morphology or in the digital domain. Hauser et al. showed in [13] and [14] that theoretically there are almost no limitations to outsourcing computation to the morphological layer. However, there are of course practical limitations as already pointed out. We would like to argue here again that these limitations are a good starting point for further investigations to understand better, where to draw the line. We also suggest that a feasible way towards a more general understanding is to look at specific tasks and to build corresponding specific morphological computation based robotic devices. When outsourcing computation to morphology we have to consider another important point. While on one hand this can be a very elegant solution, on the other hand, by doing so we fix the type of computation it can carry out. We loose the ability to be adaptive. One possibility to avoid the necessity of adaptiveness at all is to design the physical body in such a way that it already exhibits inherent robustness over a wide range. For example, a mass-spring-damper system can move back to its resting length after a perturbing force has vanished. In a dynamical system terms, we talk here about a region of attraction around a point of stability. This attractor space can be shaped by the mechanical design and is defined by the physical properties. Accordingly, for the case of repeating patterns, we can consider stable limit cycles with corresponding attractor regions around the nominal trajectory in the state space. However, there will be physical limitations for such attractor regions. Furthermore, for certain task qualitatively different responses are required, e.g., a switch from a repetitive movement to a goal driven movement (i.e., limit cycle vs. equilibrium point). A better possibility and a more general approach to overcome the problem of the missing adaptivity is to include some adaptation mechanism to change the morphology online. We call that morphosis. Such a change can include the adaptation of single parameters (e.g., the stiffness, friction coefficient, etc.), but also geometrical arrangements of body parts [26], actuators [33, 31], or sensors [19]. The EU project LOCOMORPH has investigated that line of research in the context of locomotion and built a number of novel morphosis mechanisms. The results also demonstrate how such changes can be beneficial with respect to energy efficiency and how they can increase the region of sensible parameter space over different terrain properties [32]. Typically, such changes take place on a slower time scale than the actual movement and, important in the context of this discussion, such changes are then meant to be initiated by a controller at a higher level, while at the lower level the morphological properties assure a stable movement. Another aspect of the line between abstract controller layer and mechanical body is the information flow between them. One possibility is to use the body as some nonlinear preprocessing devise to transform information in such form that the controller can deal more easily with it. An often cited example from biology is the insect eye, where the 235 Morphological Computation – The Physical Body as a Computational Resource individual rather simple visual sensors are arranged in such a nonlinear way that it counteracts (i.e., linearizing) the nonlinear effect of the optical flow, see [9]. There exists also a corresponding implementation of this principle in a robotic platform, see [19]. One could say that the morphology linearizes the input data stream. Of course, the preprocessing in the case of the insect eye is, although being nonlinear, only a simple static mapping. However, in the context of morphological computation (specifically in the case of Physical RC) a physical body can be used for much more complex tasks. For example, it could be used for preprocessing tasks that include time depending computations, i.e., the use of memory is required. This has been demonstrated in simulation results in [13], but was also shown to work in real physical platforms, see, e.g., the octopus experiments on [22]. Besides the information flow from the body to the abstract controller layer, there is also a flow into the other direction. The idea is that the high-level controller (brain) only interferes when necessary. One example has already been pointed out when we discussed the concept of morphosis, where the high level controller initiates the switch to a different behavior by adapting the morphology to the most appropriate one for the current situation (i.e., computational task). In the following section, we will elaborate this topic in more details. Switching Limit Cycles Switching Behavior Füchslin et al. discussed in [10] the idea of morphological control, i.e., computation carried out for the sake of control with the help of morphology. They described a setup where the physical part defines either a stable limit cycle or a point of equilibrium with its corresponding area of attraction. This means if the (physical) system is perturbed (in a certain maximal range) it finds its way back to the nominal behavior (either limit cycle or point of equilibrium). Now, for a different task the system might need to embody another behavior. It has to switch, for example, from one limit cycle to another, or even switch from a point of equilibrium to a limit cycle (bifurcation). Here is where the abstract control layer comes into play. It does not have to fully control the whole body, it just has to provide the appropriate input to move the system out of the actual region of attraction (of a limit cycle or equilibrium point) into a new one. One could also imaging to have a mechanism that changes the morphology of the system itself (morphosis mechanism as previously discussed). One could move a certain link of the body into a different posture. Füchslin et al. give the example of humans who change their body posture when they have to carry a heavy backpack. Another example is the "kick" we have to give our body when we change our gait patterns, e.g. to change from walking to running. In [14] Hauser et al. demonstrated such a switch in their simulations. The input to the physical body (serving as a reservoir) was a constant force to randomly chosen locations9 in the body. Depending on the amplitude of the input force the system produced autonomously one out of three different limit cycles. Note that, since the output weights are fixed after learning and we apply forces as input to a compliant body (reservoir), we 9These locations are randomly chosen in the initialization phase, when the reservoir is constructed as well. However, these locations are fixed afterwards. 236 Morphological Computation – The Physical Body as a Computational Resource Figure 2: Schematics of a possible setup to sense different environmental conditions through a soft body that serves as a reservoir. For both situations the linear setup is kept constant. On the left side the system produces autonomously a limit cycle that serves as a control signal for one degree of freedom (e.g., for the knee motor). On the right side a heavy weight is put on the body (the environment has changed) and the state of the reservoir changes. As a consequence, the system produces now a different control signal for the knee. actually change the behavior of the system (i.e., different limit cycles) by simply squeezing the body. If we squeeze it hard it produces a different limit cycle than if we squeeze it a little. One could now consider, for example, a legged robot with such a compliant body (compare Figure 2). The Physical RC setup should produce the motor position for one degree of freedom (a limit cycle for a steady locomotion) by reading out the state of the body (via internal sensors). Now, if we add a heavy weight to the back of the robot, the system should produce a different limit cycle, e.g., now for a gait with the knees more bent which is more appropriate for carrying this weight. It is important to note here that the output weights do not have to be changed. Just the different environmental conditions (heavy weight on the robot vs. the robot's weight alone) would initiated the change. One could argue the body is able to sense this difference, which is a remarkable conclusion, as this is a notion completely undiscussed in classical robot design literature. In the next section we will elaborate more on this idea of sensing through the body. However, before we do so, we would like to show this idea of switching behavior can be carried even further. Consider you have a couple of limit cycles that serve as controller trajectories for a number of active degrees of freedoms, for example, for a humanoid robot to locomote. Now if the robot stumbles and falls, it would need a completely different strategy to get up again. Instead of limit cycles, the robot would need for its degrees of freedom appropriate trajectories in its high dimensional space towards points of equilibrium (which correspond to the robot standing). For more complex robots, one could also consider some intermediate sets of equilibrium points some sort of waypoints, which define subgoals on the way to stand up. After that the robot needs again a strategy to move from this rather static posture back into the limit cycles of walking (bifurcation). The transitions between these completely different dynamic behaviors will be initiated by a high level controller in the previously discussed abstract controller layer. Note that the controller only has to initiate the transition, i.e., give it enough push (into the 237 Morphological Computation – The Physical Body as a Computational Resource right "direction") to get out of the region of attraction. Note that this control signal in general does not have to be very precise as the rest of the stable movement is "encoded" in the morphology. This also means that the controller only needs to know in which basin of attraction the system presently resides to initiate the jump into another one. Consequently, it only needs to know (and measure) the approximate state of the system. In the context of Physical RC, assuming the body is designed cleverly enough, simple (smooth) switches between different sets of output weights to produce the actuator signals might be sufficient to guide the robot through all these stages. The Smart Body Sensing Through the Body As mentioned in the previous section, the body can be used as some kind of complex sensor. This goes along with the idea presented in the section "The 'Body' and the 'Brain'" where we outlined the idea of using the body as a computational preprocessing unit. While in the machine learning setup of RC the input is some abstract signal, in a real physical systems the inputs have underlaying physical effects. For example, in the simulations presented in [13] and [14], which intent to simulate real physical systems, inputs are defined as forces. Since the physical reservoir is compliant it reacts to these forces and changes its state accordingly. Let's take a closer look at what kind of forces are considered and where they potentially can come from. For example, forces can be applied by internal actuators. This was the case in the octopus experiments in [24, 23, 22], see Figure 1c, where an attached motor moved the passive soft arm structure. In the quadruped experiments of Zhao et al. [34] the locomotion motor applied forces to the body and, therefore, introduced changes in the soft spine, which served as a physical reservoir (see Figure 1d). The input forces could also come from the environment as in the case of the "heavy weight" example, see Figure 2. Due to gravity the additional weight applies some forces to the soft body and, therefore, changes the state of the physical reservoir. Another possible way to receive input forces from the environement are external objects, which, e.g., don't allow the standard locomotion limit cycle to succeed (i.e., the robot hits an obstacle with one of its legs). Moreover, already subtle changes in the ground friction can introduce forces during locomotion, which could be then "sensed" through the body. Again, a soft body (the reservoir) would deform accordingly. A similar effect has been used in the work by Owaki et al. [25]. They showed that a mechanical communication (via the body as well via the environment) between limbs seems to be essential for robust quadruped locomotion. Furthermore, Caluwaerts et al. showed in [5] exactly the discussed sensing capabilities with a simulated tensegrity robot. They demonstrated that their robot is able to learn to distinguish different environmental conditions (flat ground, ground with nobs) by processing it through the compliant tensegrity structure during locomotion. Another input possibility are forces that are introduced by an external agent, e.g., another robot or a human guiding the passive robot. Finally, forces as input could also 238 Morphological Computation – The Physical Body as a Computational Resource be provided by an object that a soft robot wants to grasp. There would be an immediate force feedback, which could be, for example, used to differentiate various objects. From all the examples we can immediately see that sensing through a physical body is possible without the use of specific sensors. It is a very direct and fast sensing approach and no external artificial control loops are needed. However, it must be also clear that there is certain sensory information that is not directly accessible through a force interaction, for example, the information on radiation or visual input, just to name two. Nevertheless, as previously pointed out in the section on "The Power of Linear Regression" the Physical RC approach is capable to incorporate such sensory information without any problem when provided by an appropriate sensor. Finally, we would like to point to the fact that sensing through the body needs a compliant body, as well a certain extent of passive degrees of freedom (i.e., an underactuated system). Both are typical properties of soft robotic structures. Hence, we argue that soft robots are especially well suited for the Physical RC approach. Where Does the Reservoir Start and Where Does it End? In this final section, we will discuss the question where does the reservoir start and where does it end? The answer might be trivial for the abstract machine learning setup, since in that case the reservoir is explicitly defined by the designer. However, it is not that simple to find an appropriate response in setups with real physical platforms. Let's take, for example, the octopus platform from Figure 1c. At the first sight one might say that the silicone structure is the reservoir. However, this is only partially true. It is right that the (input) forces applied by the motor are transformed into a change of the dynamical state of the arm. However, there is a significant part that is contributed by the dynamics of the interaction of the arm with the environment, i.e., the water. There are nonlinear effects, drags, damping effects and so on. If we would change the property of the liquid significantly (e.g., changes the density by adding salt) we would get different responses in the sensor outputs for the same motor input signal. Note that any nonlinear effect and any temporal integration can provide potentially additional computational power as they can add to the kernel property and to the fading memory as discussed in the introduction (see also [13, 14]). To give another example, the reservoir of the quadruped robot of [34] in Figure 1d is not just the compliant spine. All other body parts contribute to the reservoir as well, since the feedback loop from the motor signal (produced by the Physical RC setup) goes through the environment, via the contact points at the feet, back to the rest of the body of the robot. As previously discussed a sufficiently big enough change in the environment would change the sensory values and, therefore, the behavior of the whole system. Next to the contribution of the interaction with the environment, actuation systems and sensor parts of the robotic structure can provide beneficial nonlinear effects to the reservoir as well. Any type of actuator as well as sensors have typically some sort of nonlinearities, e.g., saturations, memory effects, etc. They are usually perceived as disadvantages. Interestingly, in the Physical RC setup they can potentially contribute to 239 Morphological Computation – The Physical Body as a Computational Resource the computational power. Finally, when we use this setup to produce a control signal for the robot, remarkably, we can conclude that the body itself can be used as a computational resource to control itself. This means the classical separation between controller and the to-be-controlled is blurred and, therefore, we have to rethink what control means in this context. A start of this discussion has been laid out in [10]. Conclusion and Future Outlook We have discussed a number of implications when the theoretical models of [13] and [14] are implemented in real physical platforms. There are a number of remarkable conclusions, since the involved parameters from the underlaying machine learning technique (i.e., reservoir computing) relate to real physical properties. We discussed the interpretation of physical bodies as computational resources and show how the complex task to learn to emulate relevant computations can, with the help of such a body, be reduced to simple linear regression. We also provided a number of remarkable consequences, when we can use such a simple learning setup. We pointed out physical constraints and proposed how we could deal with them. We investigated the role of the body in relation to abstract controller levels and showed how a compliant physical body can be used as some sort of "full-body sensor" and as a preprocessing unit. We hope that the individual sections of this article will inspire people to do more research into that direction. We would like to refer the interested reader to the introduction of a special issue on morphological computation [15] where we have pointed out a number of research opportunities, which are also relevant for this specific Physical RC approach. Furthermore, we believe the future holds a number of interesting directions for this approach. As the research on growing and self-assembling artificial material grows more mature we will be able to build a whole range of interesting physical bodies. There exists already some theoretical work as well some simulations that show how physical bodies can be optimized to increase the performance for given tasks [16]. Also the work by Bernhardsgrütter et al. [2], where L-systems are used to control the grow of a physical reservoir, show promising results.10 If we are able to guide the growth or assembly of such real physical structures we would have a wonderful tool at our hand to build computationally powerful bodies. Although we have mostly discussed examples from robotics, the same principles can be employed for a wider range of intelligent bodies. One could apply the same framework, for example, for smart furnitures or building structures, e.g., a chair or a floor that is able to distinguish how and who is interacting with it. Another possible way to exploit the morphological computation setup is to consider smart materials, which can change their dynamic properties by applying electric or magnetic fields, or by controlling the temperature. Such physical structures would be more adaptive and resilient. For example, they could change their resonance and filtering behavior when appropriate. 10Their code is freely available on https://github.com/SoftRobotics. 240 Morphological Computation – The Physical Body as a Computational Resource So far, the discussed applications and implementations almost exclusively focused on systems governed by classical mechanics and/or continuum mechanics. However, the underlying principles apply as well to systems with dynamics governed by statistical mechanics. One important aspect of this notion is the hypothesis that also cellular control is not only enforced by genetic signals but embodies the morphology of cellular components such as, e.g., membranes as active parts in information processing. Taking the body, on the cellular or on the macroscopic level, not only as the stage on which control processes take place but as an active part of the biological computational infrastructure will influence the way how we design therapies. In summary, due to the generality of the physical reservoir computing approach a broad range of applications are possible and we believe especially an interdisciplinary approach will be highly beneficial in this context. We are excited to see what people are coming up with. Acknowledgment The project has been partially supported by the EU project FP7-PEOPLE-2013-ITN 608022 SMART-E. Helmut Hauser would like to thank Fumiya Iida for the initial idea to consider switching between more complex movements in the attractor space. Bibliography [1] L. Appeltant, M. C. Soriano, G. Van der Sande, J. Danckaert, S. Massar, J. Dambre, B. Schrauwen, C. R. Mirasso, and I. Fischer. Information processing using a single dynamical node as complex system. Nature Communications, 2:468, 2011. [2] R. Bernhardsgrütter, C. W. Senn, R. M. Füchslin, C. Jaeger, K. Nakajima, and H. Hauser. Employing L-systems to generate mass-spring networks for morphological computing. In Proceedings of International Symposium on Nonlinear Theory and its Applications (NOLTA2014), 2014. [3] S. Boyd. Volterra Series: Engineering Fundamentals. PhD thesis, UC Berkeley, 1985. [4] S. Boyd and L. Chua. Fading memory and the problem of approximating nonlinear operators with volterra series. Circuits and Systems, IEEE Transactions on, 32(11): 1150–1161, Nov 1985. ISSN 0098-4094. [5] K. Caluwaerts, M. D'Haene, D. Verstraeten, and B. Schrauwen. Locomotion without a brain: physical reservoir computing in tensegrity structures. Artificial Life, 19: 35–66, 2013. ISSN 1064-5462. doi: 10.1162/ARTL\_a\_00080. [6] K. Caluwaerts, J. Despraz, A. Işçen, A. P. Sabelhaus, J. Bruce, B. Schrauwen, and V. SunSpiral. Design and control of compliant tensegrity robots through simulation 241 Morphological Computation – The Physical Body as a Computational Resource and hardware validation. Journal of the Royal Society Interface, 11(98):20140520, 2014. [7] F. Duport, B. Schneider, A. Smerieri, M. Haelterman, and S. Massar. All-optical reservoir computing. Optics Express, 20(20):22783–22795, 2012. [8] C. Fernando and S. Sojakka. Pattern recognition in a bucket. In Advances in Artificial Life, volume 2801 of Lecture Notes in Computer Science, pages 588–597. Springer Berlin / Heidelberg, 2003. [9] N. Franceschini, J. M. Pichon, C. Blanes, and J. M. Brady. From Insect Vision to Robot Vision [and Discussion]. Philosophical Transactions Biological Sciences, 337 (1281):283–294, 1992. ISSN 09628436. [10] R. M. Füchslin, A. Dzyakanchuk, D. Flumini, H. Hauser, K. J. Hunt, R. H. Luchsinger, B. Reller, S. Scheidegger, and R. Walker. Morphological computation and morphological control: steps toward a formal theory and applications. Artificial Life, 19(1):9–34, 2013. ISSN 1064-5462. doi: 10.1162/ARTL\_a\_00079. [11] P. Hänggi. Stochastic resonance in biology. ChemPhysChem, 3(3):285–290, March 2002. doi: 10.1002/1439-7641(20020315)3:3<285::AID-CPHC285>3.0.CO;2-A. [12] H. Hauser and G. Griesbacher. Moving a robot arm by exploiting its complex compliant morphology. In R. Pfeifer, S. Hidenobu, R. M. Füchslin, H. Hauser, K. Nakajima, and S. Miyashita, editors, Proceedings of the 2nd International Conference on Morphological Computation, September 2011. [13] H. Hauser, A. Ijspeert, R. Füchslin, R. Pfeifer, and W. Maass. Towards a theoretical foundation for morphological computation with compliant bodies. Biological Cybernetics, 105:355–370, January 2011. ISSN 0340-1200. doi: 10.1007/ s00422-012-0471-0. Issue 5. [14] H. Hauser, A. Ijspeert, R. Füchslin, R. Pfeifer, and W. Maass. The role of feedback in morphological computation with compliant bodies. Biological Cybernetics, 106 (10):1–19, 2012. ISSN 0340-1200. doi: 10.1007/s00422-012-0516-4. [15] H. Hauser, H. Sumioka, R. M. Füchslin, and R. Pfeifer. Introduction to the special issue on morphological computation. Artificial Life, 19(1):1–8, 2013. [16] M. Hermans, B. Schrauwen, P. Bienstman, and J. Dambre. Automated Design of Complex Dynamic Systems. PLoS ONE, 9:e86696, 2014. ISSN 1932-6203. doi: 10.1371/journal.pone.0086696. [17] A. Isidori. Nonlinear Control Systems. Springer-Verlag GmbH, third edition, 2001. ISBN: 3-540-19916-0. [18] H. Jaeger. Adaptive nonlinear system identification with echo state networks. in S. T. S. Becker & K. Obermayer, eds, 'Advances in Neural Information Processing Systems 15', MIT Press, Cambridge, MA, pp. 593–600., 2003. 242 Morphological Computation – The Physical Body as a Computational Resource [19] L. Lichtensteiger and P. Eggenberger. Evolving the morphology of a compound eye on a robot. 1999 Third European Workshop on Advanced Mobile Robots (Eurobot'99). Proceedings (Cat. No.99EX355), 1999. doi: 10.1109/EURBOT.1999.827631. [20] W. Maass, T. Natschlaeger, and H. Markram. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11):2531–2560, 2002. [21] G. Martius, L. Jahn, H. Hauser, and V. Hafner. Self-exploration of the Stumpy robot with predictive information maximization. In A. del Pobil, E. Chinellato, E. Martinez-Martin, J. Hallam, E. Cervera, and A. Morales, editors, From Animals to Animats 13, volume 8575 of Lecture Notes in Computer Science, pages 32–42. Springer International Publishing, 2014. ISBN 978-3-319-08863-1. doi: 10.1007/ 978-3-319-08864-8_4. [22] K. Nakajima, T. Li, H. Hauser, and R. Pfeifer. Exploiting short-term memory in soft body dynamics as a computational resource. Journal of the Royal Society Interface, 11(100), November 2014. doi: 10.1098/âĂŃrsif.2014.0437. [23] K. Nakajima, H. Hauser, R. Kang, E. Guglielmino, D. G. Caldwell, and R. Pfeifer. A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm. Frontiers in Computational Neuroscience, 7:91, 2013. ISSN 1662-5188. doi: 10.3389/fncom.2013.00091. [24] K. Nakajima, H. Hauser, R. Kang, E. Guglielmino, D. G. Caldwell, and R. Pfeifer. Computing with a muscular-hydrostat system. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 1504–1511. IEEE, 2013. [25] D. Owaki, T. Kano, K. Nagasawa, A. Tero, and A. Ishiguro. Simple robot suggests physical interlimb communication is essential for quadruped walking. Journal of The Royal Society Interface, 10(78):20120669, 2013. [26] H. V. Quy, G. Ramstein, F. Casanova, L. Aryananda, M. Hoffmann, F. I. Sheikh, and H. Hauser. Gait versatility through morphological changes in a new quadruped robot. In 5th International Symposium on Adaptive Motion of Animals and Machines, 2011. [27] B. Schrauwen, D. Verstraeten, and J. Van Campenhout. An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th European Symposium on Artificial Neural Networks, pages 471–482, 2007. [28] A. Smerieri, F. Duport, Y. Paquot, B. Schrauwen, M. Haelterman, and S. Massar. Analog readout for optical reservoir computers. In Advances in Neural Information Processing Systems, pages 944–952, 2012. [29] H. Sumioka, H. Hauser, and R. Pfeifer. Computation with mechanically coupled springs for compliant robots. In The IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, September 2011. 243 Morphological Computation – The Physical Body as a Computational Resource [30] R. S. Sutton and A. G. Barto. Reinforcement Learning An Introduction. The MIT Press, first edition, 1998. [31] B. Vanderborght, A. Albu-Schaeffer, A. Bicchi, E. Burdet, D. G. Caldwell, R. Carloni, M. Catalano, O. Eiberger, W. Friedl, G. Ganesh, M. Garabini, M. Grebenstein, G. Grioli, S. Haddadin, H. Hoppner, A. Jafari, M. Laffranchi, D. Lefeber, F. Petit, S. Stramigioli, N. Tsagarakis, M. Van Damme, R. Van Ham, L. C. Visser, and S. Wolf. Variable impedance actuators: A review. Robotics and Autonomous Systems, 61:1601–1614, 2013. ISSN 09218890. doi: 10.1016/j.robot.2013.06.009. [32] H. Vu Quy, H. Hauser, D. Leach, and R. Pfeifer. A variable stiffness mechanism for improving energy efficiency of a planar single-legged hopping robot. In Advanced Robotics (ICAR), 2013 16th International Conference on, pages 1–7, Nov 2013. doi: 10.1109/ICAR.2013.6766488. [33] H. Vu Quy, L. Aryananda, F. I. Sheikh, F. Casanova, and R. Pfeifer. A novel mechanism for varying stiffness via changing transmission angle. In Proceedings IEEE International Conference on Robotics and Automation, pages 5076–5081, 2011. ISBN 9781612843865. doi: 10.1109/ICRA.2011.5980097. [34] Q. Zhao, K. Nakajima, H. Sumioka, H. Hauser, and R. Pfeifer. Spine dynamics as a computational resource in spine-driven quadruped locomotion. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013.