There are two basic approaches to the problem of induction:the empirical one, which deems that the possibility of induction depends on how theworld was made (and how it works) and the logical one, which considers the formation(and function) of language. The first is closer to being useful for induction, whilethe second is more rigorous and clearer. The purpose of this paper is to create an empiricalapproach to induction that contains the same formal exactitude as the logical approach.This requires: (a) that (...) the empirical conditions for the induction are enunciatedand (b) that the most important results already obtained from inductive logic are againdemonstrated to be valid. Here we will be dealing only with induction by elimination,namely the analysis of the experimental confutation of a theory. The result will bea rule of refutation that takes into consideration all of the empirical aspect of theexperiment and has each of the asymptotic properties which inductive logic has shown tobe characteristic of induction. (shrink)

Hi everybody! It's a great pleasure for me to be back here at the new, improved Santa Fe Institute in this spectacular location. I guess this is my fourth visit and it's always very stimulating, so I'm always very happy to visit you guys. I'd like to tell you what I've been up to lately. First of all, let me say what algorithmic information theory is good for, before telling you about the new version of it I've got.

A classic source for understanding the connections between information theory and physics, this text was written by one of the giants of 20th-century physics and is appropriate for upper-level undergraduates and graduate students. Topics include the principles of coding, coding problems and solutions, the analysis of signals, a summary of thermodynamics, thermal agitation and Brownian motion, and thermal noise in an electric circuit. A discussion of the negentropy principle of information introduces the author's renowned examination of Maxwell's demon. Concluding chapters (...) explore the associations between information theory, the uncertainty principle, and physical limits of observation, in addition to problems related to computing, organizing information, and inevitable errors. 1962 ed. 81 figures. 14 tables. (shrink)

The Shannon information function (H) has been extensively used in ecology as a statistic of species diversity. Yet, the use of Shannon diversity index has also been criticized, mainly because of its ambiguous ecological interpretation and because of its relatively great sensitivity to the relative abundances of species in the community. In my opinion, the major shortcoming of the traditional perspective (on the possible relation of species diversity with information theory) is that species need for an external receiver (the scientist (...) or ecologist) to exist and transmit information. Because organisms are self-catalized replicating structures that can transmit genotypic information to offspring, it should be evident that any single species has two possible states or alternatives: to be or not to be. In other words, species have no need for an external receiver since they are their own receivers. Therefore, the amount of biological information (at the species scale) in a community with one only species would be species, and not bits as in the traditional perspective. Moreover, species diversity appears to be a monotonic increasing function of (or S) when all species are equally probable (S being species richness), and not a function of as in the traditional perspective. To avoid the noted shortcoming, we could use 2H (instead of H) for calculating species diversity and species evenness (= 2H/S). However, owing to the relatively great sensitivity of H to the relative abundances of species in the community, the value of species dominance (= 1 − 2H/S) is unreasonably high when differences between dominant and subordinate species are considerable, thereby lowering the value of species evenness and diversity. This unsatisfactory behaviour is even more evident for Simpson index and related algorithms. I propose the use of other statistics for a better analysis of community structure, their relationship being: species evenness + species dominance = 1; species diversity × species uniformity = 1; and species diversity = species richness × species evenness. (shrink)

My aim in this paper is a modest one. I do not have any particular thesis to advance about the nature of entanglement, nor can I claim novelty for any of the material I shall discuss. My aim is simply to raise some questions about entanglement that spring naturally from certain developments in quantum information theory and are, I believe, worthy of serious consideration by philosophers of science. The main topics I discuss are different manifestations of quantum nonlocality, entanglement-assisted communication, (...) and entanglement thermodynamics. (shrink)

Abstract: Laws of computer science are prescriptive in nature but can have descriptive analogs in the physical sciences. Here, we describe a law of conservation of information in network programming, and various laws of computational motion (invariants) for programming in general, along with their pedagogical utility. Invariants specify constraints on objects in abstract computational worlds, so we describe language and data abstraction employed by software developers and compare them to Floridi's concept of levels of abstraction. We also consider Floridi's structural (...) account of reality and its fit for describing abstract computational worlds. Being abstract, such worlds are products of programmers' creative imaginations, so any "laws" in these worlds are easily broken. The worlds of computational objects need laws in the form of self-prescribed invariants, but the suspension of these laws might be creative acts. Bending the rules of abstract reality facilitates algorithm design, as we demonstrate through the example of search trees. (shrink)

We contend that if efficiency and reliability are important factors in neural information processing then distributed, not localist, representations are “evolution's best bet.” We note that distributed codes are the most efficient method for representing information, and that this efficiency minimizes metabolic costs, providing adaptive advantage to an organism.

Written by world-leading experts, this book draws together a number of important strands in contemporary approaches to the philosophical and scientific questions that emerge when dealing with the issues of computing, information, cognition and the conceptual issues that arise at their intersections. It discovers and develops the connections at the borders and in the interstices of disciplines and debates. This volume presents a range of essays that deal with the currently vigorous concerns of the philosophy of information, ontology creation and (...) control, bioinformation and biosemiotics, computational and post-computation approaches to the philosophy of cognitive science, computational linguistics, ethics, and education. http://www.amazon.ca/Computation-Information-Cognition-Gordana-Dodig-Crnkovic/dp/1847180906. (shrink)

Our work supports synchronization for binding within Phillips & Singer's “contextual field” (CF) as well as the type of its lateral interaction they propose. Both firmly agree with our “association field” (AF) and its modulatory influences (Eckhorn et al. 1990). However, the CF connections seem to produce anticorrelation among assemblies representing unrelated structures, whereas experimental evidence indicates decoupling. Finally, it is unclear how the cortex can have access to the logistic function used in the “coherent infomax” approach.

Turing machine An idealized computing device attached to a tape, each square of which is capable of holding a symbol. We write a program (a nite binary string) on the tape, and start the machine. If the machine halts with string o written at a designated place on the tape.

A new logic of partitions has been developed that is dual to ordinary logic when the latter is interpreted as the logic of subsets of a fixed universe rather than the logic of propositions. For a finite universe, the logic of subsets gave rise to finite probability theory by assigning to each subset its relative size as a probability. The analogous construction for the dual logic of partitions gives rise to a notion of logical entropy that is precisely related to (...) Claude Shannon's entropy. In this manner, the new logic of partitions provides a logico-conceptual foundation for information-theoretic entropy or information content. (shrink)

Categorical logic has shown that modern logic is essentially the logic of subsets (or “subobjects”). In “subset logic,” predicates are modeled as subsets of a universe and a predicate applies to an individual if the individual is in the subset. Partitions are dual to subsets so there is a dual logic of partitions where a “distinction” [an ordered pair of distinct elements (u, u′) from the universe U] is dual to an “element”. A predicate modeled by a partition π on (...) U would apply to a distinction if the pair of elements was distinguished by the partition π, i.e., if u and u′ were in different blocks of π. Subset logic leads to finite probability theory by taking the (Laplacian) probability as the normalized size of each subset-event of a finite universe. The analogous step in the logic of partitions is to assign to a partition the number of distinctions made by a partition normalized by the total number of ordered |U|2 pairs from the finite universe. That yields a notion of “logical entropy” for partitions and a “logical information theory.” The logical theory directly counts the (normalized) number of distinctions in a partition while Shannon’s theory gives the average number of binary partitions needed to make those same distinctions. Thus the logical theory is seen as providing a conceptual underpinning for Shannon’s theory based on the logical notion of “distinctions.”. (shrink)

Intelligent design advocate William Dembski has introduced a measure of information called “complex specified information”, or CSI. He claims that CSI is a reliable marker of design by intelligent agents. He puts forth a “Law of Conservation of Information” which states that chance and natural laws are incapable of generating CSI. In particular, CSI cannot be generated by evolutionary computation. Dembski asserts that CSI is present in intelligent causes and in the flagellum of Escherichia coli , and concludes that neither (...) have natural explanations. In this paper, we examine Dembski’s claims, point out significant errors in his reasoning, and conclude that there is no reason to accept his assertions. (shrink)

The problem of knowledge has been centred around the study of the content of our consciousness, seeing the world through internal representation, without any satisfactory account of the operations of nature that would be a pre-condition for our own performances in terms of concept efficiency in organizing action externally. If we want to better understand where and how meaning fits in nature, we have to find the proper way to decipher its organization, and account for the fact that we have (...) found codes and replicators operating at a deep levels of analysis. Informational analysis deals with units of organizational stability but it takes them for granted and leaves open the question of their origin. Patterns are used when we recognize the same configurations at different places and try to explain through their recurrence, yet to make sense of the presence of signals and counter-balancing mechanisms disseminated in nature, a hypothesis is offered to the effect that feedback signals would have a role to play in the coming about of a world that is open to new configurations and submitted to a form of stability that is more attuned to system laws than overarching unrevisable ones. (shrink)

The history of the relationship between Christian theology and the natural sciences has been conditioned by the initial decision of the masters of the "first scientific revolution" to disregard any necessary explanatory premiss to account for the constituting organization and the framing of naturally occurring entities. Not paying any attention to hierarchical control, they ended-up disseminating a vision and understanding in which it was no longer possible for a theology of nature to send questions in the direction of the experimental (...) sciences, as was done in the past between theology and many philosophically-based thought-systems. Presenting the history of some hinge-periods in the development of the Western-world sciences, this book first sets out to consider the conceptual revolution which has, in the 20th Century, related consciousness, physical laws and levels of organization, in order to show that a new chance existed then for theology. This discourse was invited to revise its language to open it up to the quest for meaning which we find on the periphery of the project of the experimental sciences. The Century-old reflection on the foundations of probability had prepared the ground for the introduction of the concept of information, at first linked to an effort aimed at maximizing the efficiency of electromagnetic communications. Taking the full measure of the questions that information theory poses to the biological sciences, this work attempts to identify the areas of convergence setting the stage for general systems theory, while it also tries to identify the insufficiencies of this recent vision and to highlight the questions left unanswered. Re-reading some of the traditional proofs of God's existence from the order of the world, relying on some pioneering insights of Ludwig von Bertalanffy and Norbert Wiener, the author brings those proofs and insights in contact with the fascinating initial project of cybernetics and the elements of a "mythical" nature which, from its inception, it could never entirely eliminate. This book ends with the confrontation between the conceptually most extended regulation factors in the history of Western thought. It articulates the poetic utopia concerned with an immediate grasp of the world in its "deictic" character with the concurrent one aimed at the domination over matter and energy expressed by technology's driving rational utopia. (shrink)

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and wherethey are fundamentally different. We discuss and relate the basicnotions of both theories: Shannon entropy, Kolmogorov complexity, Shannon mutual informationand Kolmogorov (``algorithmic'') mutual information. We explainhow universal coding may be viewed as a middle ground betweenthe two theories. We consider Shannon's rate distortion theory, whichquantifies useful (in a certain sense) information.We use the communication of information as our guiding motif, (...) and we explain howit relates to sequential question-answer sessions. (shrink)

Recent suggestions to supply quantum mechanics (QM) with realistic foundations by reformulating it in light of quantum information theory (QIT) are examined and are found wanting by pointing to a basic conceptual problem that QIT itself ignores, namely, the measurement problem. Since one cannot ignore the measurement problem and at the same time pretend to be a realist, as they stand, the suggestions to reformulate QM in light of QIT are nothing but instrumentalism in disguise.

The distinction between explanation and prediction has received much attention in recent literature, but the equally important distinction between explanation and description (or between prediction and description) remains blurred. This latter distinction is particularly important in the social sciences, where probabilistic models (or theories) often play dual roles as explanatory and descriptive devices. The distinction between explanation (or prediction) and description is explicated in the present paper in terms of information theory. The explanatory (or predictive) power of a probabilistic model (...) is identified with information taken from (or transmitted by) the environment (e.g., the independent, experimentally manipulated variables), while the descriptive power of a model reflects additional information taken from (or transmitted by) the data. Although information is usually transmitted by the data in the process of estimating parameters, it turns out that the number of free parameters is not a reliable index of transmitted information. Thus, the common practice of treating parameters as degrees-of-freedom in testing probabilistic models is questionable. Finally, this information-theoretic analysis of explanation, prediction, and description suggests ways of resolving some recent controversies surrounding the pragmatic aspects of explanation and the so-called structural identity thesis. (shrink)

Information theory offers a measure of "mutual information" which provides an appropriate measure of tracking efficiency for the naturalistic epistemologist. The statistical entropy on which it is based is arguably the best way of characterizing the uncertainty associated with the behavior of a system, and it is ontologically neutral. Though not appropriate for the naturalization of meaning, mutual information can serve as a measure of epistemic success independent of semantic maps and payoff structures. While not containing payoffs as terms, mutual (...) information places both upper and lower bounds on payoffs. This constitutes a non-trivial relationship to utility. (shrink)

The usual way to try to ground knowing according to contemporary theory of knowledge is: We know something if (1) it’s true, (2) we believe it, and (3) we believe it for the “right” reasons. Floridi proposes a better way. His grounding is based partly on probability theory, and partly on a question/answer network of verbal and behavioural interactions evolving in time. This is rather like modeling the data-exchange between a data-seeker who needs to know which button to press on (...) a food-dispenser and a data-knower who already knows the correct number. The success criterion, hence the grounding, is whether the seeker’s probability of lunch is indeed increasing (hence uncertainty is decreasing) as a result of the interaction. Floridi also suggests that his philosophy of information casts some light on the problem of consciousness. I’m not so sure. (shrink)

Increasingly encompassing models have been suggested for our world. Theories range from generally accepted to increasingly speculative to apparently bogus. The progression of theories from ego- to geo- to helio-centric models to universe and multiverse theories and beyond was accompanied by a dramatic increase in the sizes of the postulated worlds, with humans being expelled from their center to ever more remote and random locations. Rather than leading to a true theory of everything, this trend faces a turning point after (...) which the predictive power of such theories decreases (actually to zero). Incorporating the location and other capacities of the observer into such theories avoids this problem and allows to distinguish meaningful from predictively meaningless theories. This also leads to a truly complete theory of everything consisting of a (conventional objective) theory of everything plus a (novel subjective) observer process. The observer localization is neither based on the controversial anthropic principle, nor has it anything to do with the quantum-mechanical observation process. The suggested principle is extended to more practical (partial, approximate, probabilistic, parametric) world models (rather than theories of everything). Finally, I provide a justification of Ockham's razor, and criticize the anthropic principle, the doomsday argument, the no free lunch theorem, and the falsifiability dogma.", . (shrink)

Comprehensive, rigorous introduction to work of Shannon, McMillan, Feinstein and Khinchin. Translated by R. A. Silverman and M. D. Friedman.

Quantum information theory has given rise to a renewed interest in, and a new perspective on, the old issue of understanding the ways in which quantum mechanics differs from classical mechanics. The task of distinguishing between quantum and classical theory is facilitated by neutral frameworks that embrace both classical and quantum theory. In this paper, I discuss two approaches to this endeavour, the algebraic approach, and the convex set approach, with an eye to the strengths of each, and the relations (...) between the two. I end with a discussion of one particular model, the toy theory devised by Rob Spekkens, which, with minor modifications, fits neatly within the convex sets framework, and which displays in an elegant manner some of the similarities and differences between classical and quantum theories. The conclusion suggested by this investigation is that Schrödinger was right to find the essential difference between classical and quantum theory in their handling of composite systems, though Schrödinger's contention that it is entanglement that is the distinctive feature of quantum mechanics needs to be modified. (shrink)

This paper argues that Information Theoretic Redundancy (ITR) is fundamentally a composite concept that has been continually misinterpreted since the very inception of Information Theory. We view ITR as compounded of true redundancy and partial redundancy. This demarcation of true redundancy illustrates a limiting case phenomenon: the underlying metric (number of alternatives) differs only by degree but the properties of this concept differ in kind from those of partial redundancy. Several other studies are instanced which also imply the composite nature (...) of ITR. We thus provide broadly based but particular support for earlier generalized suggestions that it is the underlying calculus of Information Theory rather than the ill-named concepts themselves that provides something of a unitary language for the description of phenomena. (shrink)

given at the 2007 Formal Epistemology Workshop at Carnegie Mellon June 2nd. Good compression must track higher vs lower probability of inputs, and this is one way to approach how simplicity tracks truth.

The simplicity of a theory seems closely related to how well the theory summarizes individual data points. Think, for example, of classic curve-fitting. It is easy to get perfect data-fit with a ‘‘theory’’ that simply lists each point of data, but such a theory is maximally unsimple (for the data-fit). The simple theory suggests instead that there is one underlying curve that summarizes this data, and we usually prefer such a theory even at some expense in data-fit. In general, it (...) seems, theorizing involves looking for regularities or patterns in our experience, and such regularities are interesting to us because they summarize how our experience goes. We could list all the ravens we’ve encountered, and their colors, or we could summarize.. (shrink)

This paper addresses the problem of upgrading functional information to knowledge. Functional information is defined as syntactically well-formed, meaningful and collectively opaque data. Its use in the formal epistemology of information theories is crucial to solve the debate on the veridical nature of information, and it represents the companion notion to standard strongly semantic information, defined as well-formed, meaningful and true data. The formal framework, on which the definitions are based, uses a contextual version of the verificationist principle of truth (...) in order to connect functional to semantic information, avoiding Gettierization and decoupling from true informational contents. The upgrade operation from functional information uses the machinery of epistemic modalities in order to add data localization and accessibility as its main properties. We show in this way the conceptual worthiness of this notion for issues in contemporary epistemology debates, such as the explanation of knowledge process acquisition from information retrieval systems, and open data repositories. (shrink)

Giovanni Sommaruga (ed): Formal Theories of Information: From Shannon to Semantic Information Theory and General Concepts of Information Content Type Journal Article Pages 119-122 DOI 10.1007/s11023-011-9228-0 Authors Giuseppe Primiero, Centre for Logic and Philosophy of Science, University of Ghent, Blandijnberg 2, Ghent, 9000 Belgium Journal Minds and Machines Online ISSN 1572-8641 Print ISSN 0924-6495 Journal Volume Volume 21 Journal Issue Volume 21, Number 1.

Algorithmic information theory, or the theory of Kolmogorov complexity, has become an extraordinarily popular theory, and this is no doubt due, in some part, to the fame of Chaitin’s incompleteness results arising from this field. Actually, there are two rather different results by Chaitin: the earlier one concerns the finite limit of the provability of complexity (see Chaitin, 1974a, 1974b, 1975a); and the later is related to random reals and the halting probability (see Chaitin, 1986, 1987a, 1987b, 1988, 1989.

Understanding inductive reasoning is a problem that has engaged mankind for thousands of years. This problem is relevant to a wide range of fields and is integral to the philosophy of science. It has been tackled by many great minds ranging from philosophers to scientists to mathematicians, and more recently computer scientists. In this article we argue the case for Solomonoff Induction, a formal inductive framework which combines algorithmic information theory with the Bayesian framework. Although it achieves excellent theoretical results (...) and is based on solid philosophical foundations, the requisite technical knowledge necessary for understanding this framework has caused it to remain largely unknown and unappreciated in the wider scientific community. The main contribution of this article is to convey Solomonoff induction and its related concepts in a generally accessible form with the aim of bridging this current technical gap. In the process we examine the major historical contributions that have led to the formulation of Solomonoff Induction as well as criticisms of Solomonoff and induction in general. In particular we examine how Solomonoff induction addresses many issues that have plagued other inductive systems, such as the black ravens paradox and the confirmation problem, and compare this approach with other recent approaches. (shrink)

clusions are only probably correct. On the other hand, algorithmic information theory provides a precise mathematical definition of the notion of random or patternless sequence. In this paper we shall describe conditions under which if the sequence of coin tosses in the Solovay– Strassen and Miller–Rabin algorithms is replaced by a sequence of heads and tails that is of maximal algorithmic information content, i.e., has maximal algorithmic randomness, then one obtains an error-free test for primality. These results are only of (...) theoretical interest, since it is a manifestation of the G¨ odel incompleteness phenomenon that it is impossible to “certify” a sequence to be random by means of a proof, even though most sequences have this property. Thus by using certified random sequences one can in principle, but not in practice, convert probabilistic tests for primality into deterministic ones. (shrink)

Simple hypotheses are intrinsically attractive, and, for this reason, need to be formulated with utmost precision if they are to be testable. Unfortunately, it is hard to see how Phillips & Singer's hypothesis might be unambiguously refuted. Despite this, the authors have provided much evidence consistent with the hypothesis, and have proposed a natural and powerful extension for information theoretic approaches to learning.

This thesis is a contribution to the debate on the implications of quantum information theory for the foundations of quantum mechanics. In Part 1, the logical and conceptual status of various notions of information is assessed. It is emphasized that the everyday notion of information is to be firmly distinguished from the technical notions arising in information theory; however it is maintained that in both settings `information' functions as an abstract noun, hence does not refer to a particular or substance (...) (the worth of this point is illustrated in application to quantum teleportation). The claim that `Information is Physical' is assessed and argued to face a destructive dilemma. Accordingly, the slogan may not be understood as an ontological claim, but at best, as a methodological one. The reflections of Bruckner and Zeilinger (2001) and Deutsch and Hayden (2000) on the nature of information in quantum mechanics are critically assessed and some results presented on the characterization of entanglement in the Deutsch-Hayden formalism. Some philosophical aspects of quantum computation are discussed and general morals drawn concerning the nature of quantum information theory. In Part II, following some preliminary remarks, two particular information-theoretic approaches to the foundations of quantum mechanics are assessed in detail. It is argued that Zeilinger's (1999) Foundational Principle is unsuccessful as a foundational principle for quantum mechanics. The information-theoretic characterization theorem of Clifton, Bub and Halvorson (2003) is assessed more favourably, but the generality of the approach is questioned and it is argued that the implications of the theorem for the traditional foundational problems in quantum mechanics remains obscure. (shrink)