Norwich's Entropy Theory: how not to go from abstract to actual Lance Nizami Palo Alto, California, USA Abstract Purpose – The purpose of this paper is to ask whether a first-order-cybernetics concept, Shannon's Information Theory, actually allows a far-reaching mathematics of perception allegedly derived from it, Norwich et al.'s "Entropy Theory of Perception". Design/methodology/approach – All of The Entropy Theory, 35 years of publications, was scrutinized for its characterization of what underlies Shannon Information Theory: Shannon's "general communication system". There, "events" are passed by a "source" to a "transmitter", thence through a "noisy channel" to a "receiver", that passes "outcomes" (received events) to a "destination". Findings – In the entropy theory, "events" were sometimes interactions with the stimulus, but could be microscopic stimulus conditions. "Outcomes" often went unnamed; sometimes, the stimulus, or the interaction with it, or the resulting sensation, were "outcomes". A "source" was often implied to be a "transmitter", which frequently was a primary afferent neuron; elsewhere, the stimulus was the "transmitter" and perhaps also the "source". "Channel" was rarely named; once, it was the whole eye; once, the incident photons; elsewhere, the primary or secondary afferent. "Receiver" was usually the sensory receptor, but could be an afferent. "Destination" went unmentioned. In sum, the entropy theory's idea of Shannon's "general communication system" was entirely ambiguous. Research limitations/implications – The ambiguities indicate that, contrary to claim, the entropy theory cannot be an "information theoretical description of the process of perception". Originality/value – Scrutiny of the entropy theory's use of information theory was overdue and reveals incompatibilities that force a reconsideration of information theory's possible role in perception models. A second-order-cybernetics approach is suggested. Keywords Information theory, Perception, Communication, Systems, Cybernetics Paper type Research paper 1. Introduction The present work addresses two themes of the C:ADM2010 conference: that of "crossover processes" and that of "abstract to actual". Just how these themes are fulfilled will be recaptured in the discussion. The word "not" in the title refers to the present paper's emphasis on what not to do, knowledge that is just as important as its obverse. In 1948, two remarkable achievements were revealed; Cybernetics, or Control and Communication in the Animal and the Machine (Wiener, 1948/1961), often cited as the spark for the cybernetics movement, and "A mathematical theory of communication" (Shannon, 1974; reprint of 1948), a formulation of information transmission which is now The current issue and full text archive of this journal is available at www.emeraldinsight.com/0368-492X.htm The author's profound thanks for peer commentary to Claire S. Barnes, Faisal Kadri, and two anonymous reviewers. Thanks also to American Society for Cybernetics President Ranulph Glanville and the C:ADM Organizing Committee for accepting an earlier version of this paper for "Cybernetics: Art, Design, Mathematics", Rensselaer Polytechnic Institute, Troy, NY, USA, July 29-August 5, 2010. Finally, thanks to the Editors of the Kybernetes special issue, Ranulph Glanville (again) and Ben Sweeting. K 40,7/8 1102 Kybernetes Vol. 40 No. 7/8, 2011 pp. 1102-1118 q Emerald Group Publishing Limited 0368-492X DOI 10.1108/03684921111160331 familiar to every communications engineer. Shannon's theory was seized upon by two Harvard psychologists, Wendell R. Garner and Harold W. Hake, in "The amount of information in absolute judgments" (Garner and Hake, 1951). That paper detailed a way to apply Shannon's Information Theory to "how accurately the O [observer] perceived which of several alternative stimuli occurred on a particular presentation, or how much information theOobtained about which stimulus occurred" (Garner and Hake, 1951, p. 459). Within five years, the Garner-Hake approach was mainstream experimental psychology. It spawned hundreds of published groups of estimates of the information transmitted in absolute judgments; 47 were tabled in a recent critical review (Nizami, 2010b). The Garner and Hake (1951) approach was accepted prima facie by Professor K.H. Norwich of the University of Toronto. Indeed, Norwich desired to vastly broaden its scope to "any stage in the perceptual process", as follows. He hypothesized, from 1975 onwards, that "Our level of certainty at any stage in the perceptual process can be measured using Shannon's measure of uncertainty or entropy" (Norwich, 1981a, p. 410). To Norwich (1991a, p. 82): Information theory, because of its special structure which gives information as a function of the probabilities of a set of possible outcomes, is ideally suited to describe the perceptual process. (See the description below). Norwich (1991c, p. 180) imagined enormous gains: "Unification of the laws of sensation is the goal of the entropic or informational theory of sensation". To Norwich and his co-authors, those gains were met: "The Weber fraction, Fechner's law, Stevens's law, receptor adaptation, and many other psychophysical phenomena, find unity within this information theoretical description of the process of perception" (Norwich, 1987a, p. 286). The entropy theory also produced Teghtsoonian's law (Norwich, 1987a; repeated in Norwich, 1991c), which relates Stevens's power exponent to stimulus range (Teghtsoonian, 1971); Pieron's law for reaction time (Norwich et al., 1989; repeated in Norwich, 1991c, and in Wong and Norwich, 1996); Bloch's law, which states that stimulus duration and intensity trade off to produce a constant effect (Norwich, 1981a; Wong and Figueiredo, 2002); the "threshold law for odorants" (Norwich, 1991c); the origin of otoacoustic emissions (Norwich, 1982, 1983); Riesz's law for auditory Weber fractions (McConville et al., 1991; repeated in Wong and Norwich, 1993, 1995, 1996, in Norwich and Wong, 1997b and in Wong and Figueiredo, 2002); Weber fractions for auditory frequency (Wong and Norwich, 1993); the Stevens's law exponent for taste (Norwich, 1984a); the total number of just-noticeable-differences "making up a stimulus" (Norwich, 1984a, p. 275; repeated in McConville et al., 1991, p. 171 and in Wong and Norwich, 1995, p. 3766); equal-loudness contours (Wong and Norwich, 1995; repeated in Norwich and Wong, 1997b); and the dependence of auditory detection threshold upon stimulus duration (Wong and Figueiredo, 2002). Why loudness adapts for both ears but not for one alone was also addressed (Norwich, 2010a) along with "phantom limb" pain in amputees, and vertigo (Norwich, 2010b). No other model matches even a small fraction of all of these claims. As such, the entropy theory cries out for deserved scrutiny. 2. Quick information theory basics Norwich and co-authors had attempted to design, in the most general interpretation of the word, a mathematics of perception from the first-order cybernetics of Shannon Information Theory. To understand the entropy theory, we must therefore start with Norwich's Entropy Theory 1103 Shannon's (1974) breakthrough. Shannon described how probabilities lead to information. Considernpossible "events". When one happens, it becomes an "outcome". The outcome is uncertain whenever n . 1. Then, the events have discrete probabilities of occurring, pi, i 1⁄4 1, . . . , n. Figure 1 shows these concepts. According to Shannon, information must: . be a continuous function of the pi; . increase monotonically with n if all events are equiprobable ( pi 1⁄4 1/n); and . be the same regardless of how many steps are required to turn an event into an outcome. The amount of information available from observing an outcome is: source ðsignal Þ uncertainty ð"source information"Þ I S 1⁄4 2K Xn i1⁄41 pi log pi: ð1Þ Shannon's theory does not specify the base of the logarithm, hence the presence of K to relate a logarithm in one base to that in another. Shannon set K 1⁄4 1. When events are symbols "k": I S 1⁄4 2 k X pðkÞ log pðkÞ: ð2Þ Figure 1. "Events" and "outcomes" Event # After: The out come = one of the events (information gain = uncertainty reduction) Sources: After Shannon (1974); after a figure originally published in Nizami (2010b, p. 118) Before: events with probabilities n pi 1 0 1 2 3 4 Transmission K 40,7/8 1104 Figure 2 shows Shannon's "general communication system". Norwich et al. assumed, for simplicity's sake, that the total number of symbols transmitted and received are the same. But unintended errors in transmission can occur; symbol k, when transmitted, may be received as symbol j, with probability pj(k). Then: I t 1⁄4 I S 2 ES 1⁄4 2 k X pðkÞ log pðkÞ þ j X k X pjðkÞ log pjðkÞ; ð3Þ where: ES 1⁄4 2 j X k X pjðkÞ log pjðkÞ is the stimulus equivocation/uncertainty/entropy, here called H. I t 1⁄4 IS when what is transmitted is identically received. 3. The Entropy Theory of Perception Professor Norwich introduced various algebraic manipulations of It and ES, and introduced some conjectures involving terms s2S , m, and N 2, whose genesis need not be detailed here (the reviews are in Mori (1993) and Norwich (1993)). Norwich (1982, p. 536) eventually found a "variation of a Shannon entropy function": H ðunitless and in logarithms to base eÞ 1⁄4 1 2 ln 1 þ s2S mN 2   : ð4Þ Norwich joined Shannon Information Theory to perception through the "fundamental assumption of the entropy theory of sensation" (Norwich and Wong, 1995, p. 86), that "In the case of the isolated receptor with its sensory neuron, H is directly proportional to the frequency of impulses in the neuron" (Norwich, 1977, 1979, 1981a, 1982, 1983, p. 187, 1987a, 1991a, b; Norwich and McConville, 1991; Norwich and Wong, 1995; Wong and Norwich, 1996; Sagi et al., 2001; Wong and Figueiredo, 2002). That is, for stimulus intensity I and firing rate F(I ): Figure 2. Shannon's "general communication system" Noise source Information flow Notes: Its components: (1) "An information source which produces a message or sequence of messages to be communicated to the receiving terminal"; (2) "A transmitter which operates on the message in some way to produce a signal suitable for transmission over the channels"; (3) "The channel is merely the medium used to transmit the signal from transmitter to receiver"; (4) "The receiver ordinarily performs the inverse operation of that done by the transmitter, reconstructing the message from the signal"; and finally (5) "The destination is the person" (or thing) for whom the message is intended (Shannon, 1974, p. 5) Transmitter Message Signal (event) Received signal (outcome) Received message Noise Receiver Destination Information source Imperfect channel Norwich's Entropy Theory 1105 FðI Þ / H ; hence FðI Þ ðspikes=sÞ 1⁄4 k *H ðI Þ; k . 0: ð5Þ Norwich (1991c, p. 156) explained that "The equation F 1⁄4 kH represents a rather audacious transformation of H, which is best understood as the human attribute 'uncertainty', into a quantity F (an impulse transmission rate, say)". This k is not "the number of symbols" above. (Norwich often inexplicably duplicated or changed his notation.) Norwich then conjectured that sensation is directly proportional to neuronal firing rate, both now expressed as F 1⁄4 kH (Norwich, 1977, 1982, 1983, 1984a, b, 1987a, 1989, 1991a, b, c, 1993, 2005a; McConville et al., 1991; Mori, 1993; Wong and Norwich, 1993; Sagi et al., 2001), but perhaps with different magnitudes for k (Norwich, 1984b, p. 833). Norwich et al. describe F for sensation as "the law of sensation" (Wong and Norwich, 1995, p. 3761, 1996, p. 431; Norwich and Sagi, 2002, p. 810) or as "the psychophysical law" (Norwich, 1991c, p. 159). The entropy theory is "A model of perception based on information theoretical principles" (Norwich et al., 1989, p. 349), naturally, those of Shannon (1974). Shannon predicated his computations of information transmitted upon his "general communication system" (Figure 2). Ipso facto, the entropy theory is likewise predicated upon Shannon's "general communication system", and must therefore unambiguously differentiate "event", "outcome", "source", "transmitter", "channel", "receiver", and "destination". But does it? 4. The entropy theory's use of "event" and "outcome" Consider "event" and "outcome". Rarely did any entropy theory publication specify both. A relatively late exception was Norwich (1991a, p. 87), where: [. . .] in order to apply the equation F 1⁄4 kH to the biological process of perception we must decide which event is being perceived, what possible outcomes it may have, and what probabilities are attached to each of the outcomes. Thus, Norwich, unlike Shannon: . assumes that we receive (here, "perceive") an event, not an outcome; . assigns more than one outcome to an event; and . associates probabilities with outcomes rather than with events. Norwich (1991a, p. 87) then named the event: "Perhaps the simplest perceivable event is a sensation". He then recanted: For example, if the event is the tasting of a solution [. . .] [then it transpires that] The relevant outcomes of the tasting event are the various possible intensities [sic ] of the taste of the solution (Norwich, 1991a, p. 87). The event is now the interaction with the stimulus, and the outcome is not one of the "events", but an evoked sensation. A yet different "event" had been implied in the entropy theory all along (Norwich, 1981a, 1987a): "We can envisage a steady sensory stimulus as a stationary stochastic sequence of microscopic sensory events" (Norwich and Wong, 1997a, p. 164). That is, as Norwich and Wong explained, events are different stimulus intensities. The receptor sampled them to compute H: as Wong and Figueiredo (2002, p. ICAD02-2) later noted: K 40,7/8 1106 The basic premise for calculating H is to assume that the receptor samples the sensory signal [stimulus] to estimate the magnitude of the input [intensity]. The uncertainty in signal magnitude is attributed to the variability or fluctuation in the signal at the receptor level. Figure 3 (after Norwich (1981a, 1987a)) illustrates these concepts. Figure 3. The physiological response to a sensory stimulus in the entropy theory t1 t2 t3 Time Receptor + neuronStimulus Notes: The stimulus, envisioned as a cloud of particles, is macroscopically constant; the larger box always contains the same number of particles (here, 25); within the larger box is a smaller box, whose particle population fluctuates in thermodynamic equilibrium with its surrounding, the number of particles within the smaller box (open circles) varying from eight at time t1 to four at time t2 to six at time t3, etc.; at each such instant, parallel, identical receptor-neuron units faultlessly count the number of particles within the smaller box, sending that count to the brain Sources: After Figure 1 of Nizami (2010a); permission applied for reproduction Brain Information flow Norwich's Entropy Theory 1107 Overall, the entropy theory's interpretations of "event" and "outcome" were contradictory, and did not follow Shannon's definitions. 5. The entropy theory's use of "source", "transmitter", "channel", "receiver", and "destination" 5.1 Source and/or transmitter Information, early in the entropy theory, was "transmitted" by "isolated animal neurons" (Norwich, 1979), primary afferents serving sensory receptors. Then Norwich (1987a, p. 290) inexplicably reversed, speaking of "information transmitted to the receptor", i.e. the source and transmitter precede the receptor and hence the afferents. The same paper then mentioned "the maximum information transmissible by that sensory modality" (p. 290), i.e. the sensory modality 1⁄4 transmitter. Yet, later therein, Norwich mentioned "the information transmitted by a stimulus" (p. 297), so that stimulus 1⁄4 transmitter. And the story continued to change without explanation. Norwich (1989, p. 285) referred to "the information transmitted from stimulus to perceiver", i.e. stimulus 1⁄4 source and "perceiver" 1⁄4 receiver. McConville et al. (1991, p. 171) did likewise, but Norwich and McConville (1991) declared that "we regard the environment as a transmitter of information, and the sensory receptor as the receiver", which "may receive only as much information as the stimulus transmits" (Norwich and McConville, 1991, pp. 151-2; italics added). Yet later therein, Norwich and McConville (1991, p. 154) described the amount of information "transmitted by the cockroach spine" (i.e. spinal neuron 1⁄4 transmitter), then stated that "The cockroach mechanoreceptor, therefore, transmits nearly the same quantity of information as mammalian sensory receptors" (i.e. sensory receptor 1⁄4 transmitter). Receptors were still transmitters in Norwich and Wong (1995, p. 84): "The [Entropy] model utilizes a property that all sensory receptors share, the capacity to transmit information". Two lines later, however, Norwich and Wong mention "Information transmitted by the stimulus", i.e. stimulus 1⁄4 transmitter. Several lines further on, they state "We regard the stimulus as a signal", that is, the stimulus is "the stimulus signal" (Norwich and Wong, 1995, p. 88), the thing to be transmitted. On the same page, then, Norwich and Wong imply at least three conflicting interpretations of "transmitter". Inexplicably, Wong and Figueiredo (2002) named the afferent neurons as the transmitters. 5.2 Channel In Norwich (1978) and sometime thereafter, the visual channel was the whole eye including the receptors. "The sensory receptor forms part of a channel" (Norwich, 1983, p. 175). Later yet, however, "the train of photons proceeding from the object to the receptor constitutes the channel" (Norwich, 1984b, p. 832). Further therein (p. 834), Norwich stated that information is transmitted to the perceiver (also Norwich (1979), in Norwich (1991a, p. 84) as "to the observer"), i.e. the channel is now an unnamed medium linking source and perceiver. Post-1984, Norwich and colleagues did not explicitly identify "channel" at all. Nonetheless, in Norwich and McConville (1991), the channels perforce were afferent neurons, because the transmitter was a spinal neuron, and then it was a receptor, both of which would connect to afferents. K 40,7/8 1108 5.3 Receiver Norwich (1977) named the sensory receptor as the receiver (Norwich, 1984a, pp. 269, 273, 1989, p. 356). However, in Norwich (1989), the whole human was implied to be the receiver (see above); later yet, in Norwich and McConville (1991), the receiver went unnamed, but was perforce some neuron(s), as the transmitter was at first a spinal neuron (see above), and then a receptor. 5.4 Destination "Destination" never appeared in the entropy theory. 5.5 Summary Table I lists Norwich and colleagues' evident interpretations of "source", "transmitter", "channel", and "receiver". These changed in a logic-free manner that evolved towards no consistent description of the Shannon "general communication system". Indeed, Shannon's terms were entirely absent from nine of the 11 papers published by Norwich and colleagues since 1991. 6. Overall summary Shannon's Information Theory depends upon Shannon's "general communication system", which unambiguously uses "event", "outcome", "source", "transmitter", "channel", "receiver", and "destination". Shannon's Information Theory also allegedly forms the basis of Norwich and colleagues' Entropy Theory of Perception (1975-present). The latter should therefore employ Shannon's concepts unambiguously. Instead, it contains a confusing gamut of interpretations, sometimes several contemporaneously, sometimes none at all. Nonetheless, Norwich and colleagues offered no formal alternative to the Shannon system. Altogether, the entropy theory is not "informational". One contributor, Wong (2007, p. 2), lately noted that: [. . .] little justification (beyond the simple analogy of the senses as a communications system) was provided for why information underlies the process of perception [in the Entropy Theory] and why this result is universal. But Wong missed the point: "simple analogies" are insufficient. This problem is perpetually central to any application of Shannon Information Theory. Other applications were made of Shannon Information Theory to perception (Baird, 1970a, b), which proved incompetent (MacRae, 1972). Understanding perception requires a holistic approach (Nizami, 2010b). 7. Discussion/conclusions 7.1 Regarding jargon Some readers may object to the jargonistic nature of the above review. However, the jargon is inherent to information theory, and thus to the manner in which Norwich and co-authors misused that theory. Indeed, the jargonistic nature of the entropy theory betrays its lack of generality and hence its failure to apply as it was claimed to apply. 7.2 C:ADM2010 conference themes The present work addresses both of the C:ADM2010 conference themes of "crossover processes" and "abstract to actual". "Abstract to actual" refers to the Norwich et al. Norwich's Entropy Theory 1109 R ef er en ce S ou rc e T ra n sm it te r C h an n el R ec ei v er N or w ic h (1 97 6) – P ri m ar y af fe re n t n eu ro n P ri m ar y af fe re n t n eu ro n – N or w ic h (1 97 7) – – – R ec ep to r N or w ic h (1 97 8) – – W h ol e ey e R od or co n e N or w ic h (1 97 9) – P ri m ar y af fe re n t n eu ro n – – N or w ic h (1 98 1a ) – T h e p er ce p tu al p ro ce ss – – N or w ic h (1 98 1b ) – – – – N or w ic h (1 98 2) – – – – N or w ic h (1 98 3) – – R ec ep to r – N or w ic h (1 98 4a ) – S ti m u lu s – R ec ep to r N or w ic h (1 98 4b ) – – P h ot on st re am – N or w ic h (1 98 7a ) – T h e se n so ry m od al it y – – N or w ic h (1 98 7a ) – S ti m u lu s – – N or w ic h (1 98 7b ) "O u ts id e w or ld " R ec ep to r – "B ra in an d n er v ou s sy st em " N or w ic h (1 98 7b ) – – – R ec ep to r N or w ic h (1 98 9) "O b je ct iv e w or ld " – – "T h e p er ce iv er " N or w ic h et a l. (1 98 9) – – – R ec ep to r N or w ic h (1 99 1a ) – "T h e p er ce p t" – – N or w ic h (1 99 1a ) – T h e ev en t – – N or w ic h (1 99 1b ) – – – – N or w ic h (1 99 1c ) – – – – N or w ic h an d M cC on v il le (1 99 1) S ti m u lu s T h e en v ir on m en t – – N or w ic h an d M cC on v il le (1 99 1) – S ti m u lu s – – N or w ic h an d M cC on v il le (1 99 1) – S p in al n eu ro n – – N or w ic h an d M cC on v il le (1 99 1) – R ec ep to r – – M cC on v il le et a l. (1 99 1) S ti m u lu s – – W h ol e h u m an N or w ic h (1 99 2) – – – – W on g an d N or w ic h (1 99 3) – – – – N or w ic h an d W on g (1 99 5) – R ec ep to r – – N or w ic h an d W on g (1 99 5) – S ti m u lu s – – W on g an d N or w ic h (1 99 5) – – – – W on g an d N or w ic h (1 99 6) – – – – N or w ic h an d W on g (1 99 7a ) – – – – N or w ic h an d W on g (1 99 7b ) – – – – S ag i et a l. (2 00 1) – – – – W on g an d F ig u ei re d o (2 00 2) – P ri m ar y af fe re n t n eu ro n – – W on g (2 00 7) – – – – N or w ic h (2 01 0a ) – – – – N or w ic h (2 01 0b ) – – – – N o te s : M u lt ip le li st in g s of a p ap er in d ic at e d if fe re n t u se s th er ei n ; a d as h in d ic at es th at n o p ar ti cu la r d efi n it io n co u ld b e in fe rr ed Table I. Uses of "source", "transmitter", "channel", and "receiver" in the Entropy Theory of Perception K 40,7/8 1110 attempt to apply an abstraction (Shannon Information Theory and its "general communication system") to an actual process (perception, a result of "sensory systems"). Norwich et al.'s entropy theory fails to link abstract to actual, and in so doing it also fails to "crossover between fields", i.e. to make an engineering model (Shannon Information Theory) work in perceptual psychology. 7.3 Why does the entropy theory fail? Norwich et al.'s entropy theory fails. Why? It depends upon Shannon Information Theory, a first-order cybernetics model. In such a model, information transmitted is computed by an external observer who stands apart from the system, an engineer. That is, the Shannon general communication system does not compute its own information. But Norwich et al. claim that to perceive requires information computation. Such computation must thus be inherent to the perceptual "system". The perceiver is the presumed gainer of "information", hence the perceiver is formally the observer, and the latter must constitute one or more parts of the Shannon general communication system. But which part or parts? Norwich, for example, sometimes equated the sensory receptor itself to the perceiver (Norwich, 1987b), rendering redundant the rest of the ascending nervous system (including the brain). Sometimes, he called the observer the perceiver (Norwich, 2005b, p. 249). Regardless, the Shannon system (and its mathematics) is not set up for an internal observer (unlike the goal of "second-order" cybernetics). Hence the entropy theory's welter of different interpretations of what the elements of the Shannon system actually are in man or animal. Norwich offers no clear interpretation; he cannot. An inseparable point is that physically, perception involves interaction with the stimulus, interaction that can transform the stimulus. For example, in vision the eye absorbs light, and each light-photon's energy is converted to other forms of energy. In the sense of taste, tastant molecules are adsorbed upon the taste receptors, and can be altered within the oral cavity. In any case, they can be absorbed by, and may be destroyed within, the digestive system. In audition, the air pressure waves which we perceive as "sound" are damped in amplitude at the points where they interact with the eardrum. And so on, and so forth. Norwich (2010b) himself admitted as much, but never explained how Shannon Information Theory was supposed to cope with this phenomenon. In perception, the stimulus and the perceiver are both changed. The observer, who is the perceiver, is changed by their interaction with the stimulus. The stimulus is "a difference that makes a difference". But in the Shannon general communication system, the operation of no part of the system is altered by the stimulus. That is because Shannon Information Theory describes only amounts of information, not the character of the information; Shannon Information Theory does not account for the received information having meaning, hence Shannon Information Theory does not allow for changes in the system. Overall, the moral of the story is that second-order cybernetics should be respected when addressing perception. Few perceptionists, however, seem aware of second-order cybernetics. 7.4 Was the entropy theory useful? The entropy theory is cited in the perception literature, but largely by persons of limited mathematical credentials who simply repeat the theory's ideas, perhaps not realizing its Norwich's Entropy Theory 1111 problems (or perhaps simply ignoring them). Those problems reach far beyond the ambiguities revealed here (Nizami, 2010b). But one could object that models prove incorrect anyway; they only need be useful. Was the entropy theory useful, i.e. in encouraging new theories and new experiments? The entropy theory inspired no new theories. Indeed it proves useful to theory only in the present ironic context, that is, as a refutation. It inspired only one experiment, which actually disproved one of the theory's basic tenets (Nizami, 2010b). 7.5 A connection of Shannon Information Theory to art? The issue of art is particularly relevant. Indeed, the major inspiration for Norwich et al.'s Entropy Theory of Perception was an earlier attempt by someone else to connect Shannon Information Theory to music and to art, as follows. Norwich himself never explained what inspired his theory. Further, the citations in his papers are no help; most are autobiographical, and indeed, his publications are noteworthy for their paucity of non-autobiographical supporting references. Nonetheless, there are a few lone individuals whom Norwich et al. cite more than once in the entropy theory. Among them is Abraham Moles (Norwich, 1983, 1984a, b, 1991c, 2005a, b). Specifically, Norwich cites Moles's (1966) book Information Theory andEsthetic Perception. This book has enjoyed several hundred other citations, some quite recently, but few by scientists, as suggested by an online search. Regardless, Norwich clearly admires Moles and his book. To discover what Norwich found so compelling seems important, but is easier said than done. On the first page of his Introduction, Moles already uses the jargon of Shannon Information Theory. Some might think this a propitious start. Moles (1966, p. 1) implies the human subject to be the "receiver" who receives "messages" from "the inert world or from other individuals". On the very next page he introduces art, describing it as "a creator of sensations". He then notes that: It will be the goal of the present work to try to integrate in a coherent fashion the essential concepts which have issued from the science of communication and more precisely from the body of doctrine known as information theory, with our vision of the world, that is, our perception, particularly in the domain, hitherto neglected, of esthetics and the psychology of perception (Moles, 1966, p. 2). A bold aim indeed. The information theory mentioned is that of Shannon, a point not at first apparent. Further, the specifics of the Shannon system of interest are never clearly specified. One can, nonetheless, immediately recognize the origin of Norwich's grandiose desire to explain all of intensity perception using information theory (more on this later). Indeed, Norwich's notion that perception occurs through sampling (Figure 3) may be inferred from Moles's (1966, p. 3) remark that "All behavior, present or future, of the individual may be described with a degree of precision equal to that of the description of a physicochemical system". Nearby, Moles (1966, p. 5) states that "Noise limits the individual's apprehension of the exterior world", a notion that is core to the entropy theory (Norwich et al., various). Moving on to his first chapter, Moles (1966, p. 7) elaborates upon his jargon: The individual receives messages from this [surrounding] environment through various channels: visual, aural, tactile, etc. The term "channel" is applied to any material system which conveys a message from a transmitter to a receiver. Moles goes on to classify channels as "natural channels" such as vision and hearing, in which man is the "receiver", and "technical channels", in which machines receive K 40,7/8 1112 electrical signals. Moles (1966, p. 8) notes that in natural channels, "it is often difficult to separate the brain from the sensory receptors", an understatement that underscores the differing interpretations in the present Table I. For much of the rest of his book, Moles insists on calling the human receiver the "receptor". Moles (1966, p. 8) then swiftly connects information theory to art, as follows: "A painting, a drawing, or a photograph gives us messages in two dimensions". He also mentions animated cartoons and movies as being messages, along with speech and music. This description is vague. Presumably in an effort to make it clear, Moles (1966, p. 9) offers the following, which sets the tone for the rest of the book: A message is a finite, ordered set of elements of perception drawn from a repertoire and assembled in a structure. The elements of the repertoire are defined by the properties of the receiver. A specific study will reveal the nature of the elements and the repertoire of artificial [i.e., technical] channels. For messages of natural channels directed to the sense organs, the elements of the repertoire are enumerated by the various departments of psychophysiology. This kind of obtuse pedantry typifies the rest of the book, which consists largely of Moles's interpretations, rarely supported by references, of what the various repertoires, structures, elements, properties, and "departments" actually are. The rest of his Chapter 1 reviews information theory. In his second chapter, Moles (1966, p. 65) forays into the notion of channel capacity and "the receptor's capacity to know". He also introduces "sonic events: thuds, musical notes, etc." (Moles, 1966, p. 70), which remain one of his favorite subjects. He states, as if proven, that "One of the fundamental characteristics of the human receptor is the existence of a maximum limit to the flow of perceptible information" (Moles, 1966, p. 74), and furthermore that "When this maximum flow is exceeded, the individual selects, with the aid of criteria derived from his previous experience, forms from the message presented to him" (Moles, 1966, p. 74). All of the reasoning involved is dense and extremely difficult to follow and cannot be synopsized here. Moles's next chapter differentiates "forms" from "noises". He states that "We may assume, as the key to the perception of forms, that the human receptor [sic ] has direct perception of a signal's autocorrelation" (Moles, 1966, p. 101). Moles (1966, p. 101) also declares that: The rate of information of a message is determined by the structures that the receptor perceives in the message. These structures are created by memory, which summarizes the set of messages the individual has already received in statistical rules or in symbols. And further, "The receptor is thus a developing (learning) system. Each message modifies the receptor's capacity to receive succeeding messages". Here we can see an attempt to relate the need for knowledge of the frequency of possible events in order to compute the Shannon information, although the latter should by no means affect "the receptor's capacity to receive succeeding messages" per se, only the capacity of someone (not necessarily the "receptor") to compute their information content. This is only one example of the many confusions in Moles's text, in which he attempts to broaden Shannon Information Theory, which we may think of as a "special theory of information transmission", into a "general theory of information transmission". In brief, Moles attempts to do for information theory what A. Einstein did for physics when he broadened "special relativity" into "general relativity". Of course, Einstein Norwich's Entropy Theory 1113 supplemented the algebra; Moles does not. Nonetheless, according to Moles (1966, p. 99), he has succeeded: Thus, the scope of information theory is notably enlarged. Originally the theory treated only material messages, always in the identical form of the mechanical receptor's code. But this framework is too rigid for experimental esthetics and for perceptual theory, both of which must essentially take account of individual variations [in human experience, knowledge, etc.]. Ironically, Moles is correct in that Shannon's theory does not account for "experimental esthetics and for perceptual theory", but he is wrong in thinking that he (or anyone else) has yet solved that problem, as we will see. In Moles's next chapter, art and esthetics finally reappear after a long delay: An art is exactly defined by the set of rules it follows. The role of esthetics, considered as a science, is to enumerate those rules and link them with universal laws of perception (Moles, 1966, p. 105). Thus, Moles connects art to "universal laws of perception" by way of esthetics. After a lengthy discourse on sonic objects, Moles connects art to information theory in the next chapter. This is Moles's hypothesis, which the reader has had to wait 127 pages to read: When an individual rereads a book that he knows, even if he wrote it himself, when he rehears a symphony whose themes he knows by heart, even if he conducted it, still he never knows the message so exactly, so perfectly, and in such detail that he gains absolutely nothing new from it. It is never totally banal to him. There is always something to be drawn from it, a residual information to be gleaned, because a human being's memory cannot exhaust a message of such scale (Moles, 1966, p. 127). Note the very loose usage of the word "information". Moles's next major statement in this progression has also to be quoted in order to grasp its irony and to avoid any accusation of misrepresentation: We [sic ] propose the existence of two types of information in messages generally. These types of information depend on how an observer outside the transmission channel groups elements of the message in order to relate them to repertoires. (a) Semantic information, having a universal logic, structured, articulable, translatable into a foreign language, serves in the behaviorist conception to prepare actions. (b) Instead of to a universal repertoire, esthetic information, which is untranslatable, refers to the repertoire of knowledge common to the particular transmitter and the particular receptor (Moles, 1966, p. 129). Note that the use of humans as transmitters and receivers has, by this point, wiped away any remaining relation to the "technical channel" described by Shannon (1974). But where is the alternative, the information theory for the "natural channel"? As examples of semantic information, Moles (1966, p. 130) mentions "a military order, an electrical circuitry diagram, a coded message, instructions in case of fire, a technical manual, a musical score, etc.". In contrast, Moles (1966, p. 132) declares that: [. . .] esthetic information is randomized and specific to the receptor, since it varies according to his repertoire of knowledge, symbols, and a priori structurings, which in turn relate to his sociological background. Further, "It is very poorly known and difficult to measure" (Moles, 1966, p. 132). In short, esthetics is amenable to analysis by information theory, but is not. K 40,7/8 1114 But let's not be hasty. Regarding art, Moles (1966, p. 133) explains that: The original parts of the esthetic message express statistically what gives value to a particular painting within a given style; they are what traditional pictorial esthetics calls by the vague terms personality of the picture, mastery, originality, etc. Thus, it sounds like a theory of esthetics is imminent. But the book then disintegrates into a series of complex diagrams and jargonistic digressions, with no information theory algebra in sight, and no explanation of the system at work. One notable anomaly is a puzzling figure showing "Approximate distribution of semantic and esthetic information over the range of acoustical frequencies for music", which appears without explanation or supporting references. After yet another chapter, entitled "Multiple messages and structural esthetics", filled with complicated box-and-arrow diagrams, Moles finally reaches conclusions. There he declares (Moles, 1966, p. 199) that "The esthetic message has symbols unknown to us and rules often little known; it is untranslatable and unique". Thus, he seems to confess, once and for all, that there is no relation of information theory to esthetic perception. The present writer found Moles's book one of the most excruciating reads in his experience. It resembles the work of Norwich et al., in that both are filled with unsupported declarations and assumptions, as well as spurious allusions to men of scientific genius. All this can be independently confirmed. Anyone who considers the present assessment to be harsh is invited to read the Keats (1966) review of Moles (1966). Keats concludes that "Moles, like many before him, has failed to see that a relabelling of old problems does not necessarily increase our knowledge or understanding of them". The same can fairly be said of Norwich et al., whose Entropy Theory of Perception bears numerous striking parallels to Moles (1966), far more than described here, leaving little doubt that Moles (1966) was the (albeit uncredited) origin of Norwich's ideas. References Baird, J.C. (1970a), "A cognitive theory of psychophysics. I. Information transmission, partitioning, and Weber's law", Scandinavian Journal of Psychology, Vol. 11 No. 1, pp. 35-46. Baird, J.C. (1970b), "A cognitive theory of psychophysics. II. Fechner's law and Stevens' law", Scandinavian Journal of Psychology, Vol. 11 No. 2, pp. 89-102. Garner, W.R. and Hake, H.W. (1951), "The amount of information in absolute judgements", Psychological Review, Vol. 58 No. 6, pp. 445-59. Keats, W.L. (1966), "The Joel E. Cohen translation of Abraham Moles's Information Theory and Esthetic Perception", Harvard Crimson, March 18, available at: www.thecrimson.com/ article/1966/3/18/the-joel-e-cohen-translation-of/ McConville, K.M.V., Norwich, K.H. and Abel, S.M. (1991), "Application of the entropy theory of perception to auditory intensity discrimination", International Journal of Biomedical Computing, Vol. 27 Nos 3/4, pp. 157-73. MacRae, A.W. (1972), "Information transmission, partitioning and Weber's law: some comments on Baird's cognitive theory of psychophysics", Scandinavian Journal of Psychology, Vol. 13 No. 1, pp. 73-80. Moles, A. (1966), Information Theory and Esthetic Perception, University of Illinois Press, Urbana, IL (transl. Cohen, J.E.). Norwich's Entropy Theory 1115 Mori, S. (1993), "Toward a unified theory of psychophysics: Norwich's entropy theory of perception", Japanese Psychological Review, Vol. 36 No. 2, pp. 244-64. Nizami, L. (2010a), "Fundamental flaws in the derivation of Stevens' Law for taste within Norwich's Entropy Theory of Perception", in Korsunsky, A.M. (Ed.), Current Themes in Engineering Science 2009, American Institute of Physics, Melville, NY, pp. 150-64. Nizami, L. (2010b), "Interpretation of absolute judgments using information theory: channel capacity or memory capacity?", Cybernetics & Human Knowing, Vol. 17 Nos 1/2, pp. 111-55 (doi: 10.3389/fphys.2010.00017). Norwich, K.H. (1976), "An hypothesis on the processing of information by sensory receptors", paper presented at the 11th International Conference on Medical and Biological Engineering, Ottawa. Norwich, K.H. (1977), "On the information received by sensory receptors", Bulletin of Mathematical Biology, Vol. 39 No. 4, pp. 453-61. Norwich, K.H. (1978), "An hypothesis on information, memory, and perception", Medical Hypotheses, Vol. 4 No. 2, pp. 156-64. Norwich, K.H. (1979), "The information content of a steady sensory stimulus", paper presented at the 12th International Conference on Medical and Biological Engineering and 5th International Conference on Medical Physics, Jerusalem. Norwich, K.H. (1981a), "The magical number seven: making a 'bit' of 'sense'", Perception & Psychophysics, Vol. 29 No. 5, pp. 409-22. Norwich, K.H. (1981b), "Uncertainty in physiology and physics", Bulletin of Mathematical Biology, Vol. 43 No. 2, pp. 141-9. Norwich, K.H. (1982), "Perception as an active process", Mathematics & Computers in Simulation, Vol. 24 No. 6, pp. 535-9. Norwich, K.H. (1983), "To perceive is to doubt: the relativity of perception", Journal of Theoretical Biology, Vol. 102 No. 2, pp. 175-90. Norwich, K.H. (1984a), "The psychophysics of taste from the entropy of the stimulus", Perception & Psychophysics, Vol. 35 No. 3, pp. 269-78. Norwich, K.H. (1984b), "Why the eye may be found to be a source of light", paper presented at the Sixth International Congress of Cybernetics & Systems, World Organisation of General Systems & Cybernetics, Paris, September. Norwich, K.H. (1987a), "On the theory of Weber fractions", Perception & Psychophysics, Vol. 42 No. 3, pp. 286-98. Norwich, K.H. (1987b), "The physics of prayer and the origin of the universe", Conservative Judaism, Vol. 40 No. 2, pp. 14-19. Norwich, K.H. (1989), "The Fechner-Stevens law is the law of transmission of information", Behavioral & Brain Sciences, Vol. 12 No. 2, p. 285. Norwich, K.H. (1991a), "On the fundamental nature of perception", Acta Biotheoretica, Vol. 39 No. 1, pp. 81-90. Norwich, K.H. (1991b), "The psychophysical response is equal to the potential stimulus information", paper presented at the 7th Annual Meeting of the International Society for Psychophysics, Durham, NC, October 17-21. Norwich, K.H. (1991c), "Toward the unification of the laws of sensation: some food for thought", in Lawless, H.T. and Klein, B.P. (Eds), Sensory Science Theory and Applications in Foods, Marcel Dekker, New York, NY, pp. 151-83. K 40,7/8 1116 Norwich, K.H. (1992), "Context effects in the entropic theory of perception", Behavioral & Brain Sciences, Vol. 15 No. 3, pp. 578-9. Norwich, K.H. (1993), Information, Sensation, and Perception, Academic Press, Toronto. Norwich, K.H. (2005a), "Physical entropy and the senses", Acta Biotheoretica, Vol. 53 No. 3, pp. 167-80. Norwich, K.H. (2005b), "The legacy of Abraham Moles and Erwin Schrödinger in psychophysics", paper presented at the 21st Annual Meeting of the International Society for Psychophysics, Traverse City, MI, October 19-22. Norwich, K.H. (2010a), "A mathematical exploration of the mystery of loudness adaptation", Bulletin of Mathematical Biology, Vol. 72 No. 2, pp. 298-313. Norwich, K.H. (2010b), "Le Chatelier's principle in sensation and perception: fractal-like enfolding at different scales", Frontiers in Fractal Physiology, Vol. 1, p. 17. Norwich, K.H. and McConville, K.M.V. (1991), "An informational approach to sensory adaptation", Journal of Comparative Physiology, Vol. A168 No. 2, pp. 151-7. Norwich, K.H. and Sagi, E. (2002), "Deriving the loudness exponent from categorical judgments", Perception & Psychophysics, Vol. 64 No. 5, pp. 804-14. Norwich, K.H. and Wong, W. (1995), "A universal model of single-unit sensory receptor action", Mathematical Biosciences, Vol. 125 No. 1, pp. 83-108. Norwich, K.H. and Wong, W. (1997a), "Sensory function in extraterrestrial beings", Annales de la Fondation Louis de Broglie, Vol. 22 No. 3, pp. 161-8. Norwich, K.H. and Wong, W. (1997b), "Unification of psychophysical phenomena: the complete form of Fechner's law", Perception & Psychophysics, Vol. 59 No. 6, pp. 929-40. Norwich, K.H., Seburn, C.N.L. and Axelrad, E. (1989), "An informational approach to reaction times", Bulletin of Mathematical Biology, Vol. 51 No. 3, pp. 347-58. Sagi, E., Norwich, K.H. and Kunov, H. (2001), "Loudness encoding at the auditory nerve", Canadian Acoustics, Vol. 29 No. 3, pp. 36-7. Shannon, C.E. (1974), "A mathematical theory of communication", in Slepian, D. (Ed.) Key Papers in the Development of Information Theory, IEEE Press, New York, NY, pp. 5-18 (Reprint of Shannon, C.E. (1948), "A mathematical theory of communication", Bell System Technical Journal, Vol. 27 No. 3, pp. 379-423.). Teghtsoonian, R. (1971), "On the exponents in Stevens' law and the constant in Ekman's law", Psychological Review, Vol. 78 No. 1, pp. 71-80. Wiener, N. (1948/1961), Cybernetics or Control and Communication in the Animal and the Machine, Wiley, New York, NY. Wong, W. (2007), "On the physical basis of perceptual information", paper presented at the 51st Meeting of ISSS, Tokyo, August 5-10. Wong, W. and Figueiredo, S. (2002), "On the role of information and uncertainty in auditory thresholds", paper presented at the 2002 International Conference on Auditory Display, Kyoto, July 2-5. Wong, W. and Norwich, K.H. (1993), "Frequency and the sensation of sounds", Canadian Acoustics, Vol. 21 No. 3, pp. 133-4. Wong, W. and Norwich, K.H. (1995), "Obtaining equal loudness contours from Weber fractions", Journal of the Acoustical Society of America, Vol. 97 No. 6, pp. 3761-7. Wong, W. and Norwich, K.H. (1996), "Weber fraction and reaction time from the neural entropy", paper presented at the 12th Annual Meeting of the International Society for Psychophysics, Padua, October 19-22. Norwich's Entropy Theory 1117 Further reading Norwich, K.H. (1975), "Information, memory, and perception", Institute of Biomedical Engineering Report, Vol. 17, University of Toronto, Toronto. About the author Lance Nizami has degrees in Physics (BSc), Physiology Theory (MSc), and Perceptual Psychology (PhD), all from University of Toronto, where he was also granted acceptance to the Architecture program (declined in favor of graduate study). He is a citizen of the UK, Canada, and the USA, and has been a Postdoctoral Research Fellow and Research Associate at Boys Town National Research Hospital, Omaha, Nebraska. He is presently an unfunded Independent Research Scholar, expanding and publishing a great deal of work left over from his degree studies. The focus of his work has been hearing research, both through experiments and through mathematical modeling. He maintains a longstanding interest in the arts and design and is a published poet. Lance Nizami can be contacted at: nizamii2@att.net K 40,7/8 1118 To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints