Entropy is ubiquitous in physics, and it plays important roles in numerous other disciplines ranging from logic and statistics to biology and economics. However, a closer look reveals a complicated picture: entropy is defined differently in different contexts, and even within the same domain different notions of entropy are at work. Some of these are defined in terms of probabilities, others are not. The aim of this chapter is to arrive at an understanding of some of the (...) most important notions of entropy and to clarify the relations between them, After setting the stage by introducing the thermodynamic entropy, we discuss notions of entropy in information theory, statistical mechanics, dynamical systems theory and fractal geometry. (shrink)

Daniel R. Brooks and E. O. Wiley have proposed a theory of evolution in which fitness is merely a rate determining factor. Evolution is driven by non-equilibrium processes which increase the entropy and information content of species together. Evolution can occur without environmental selection, since increased complexity and organization result from the likely capture at the species level of random variations produced at the chemical level. Speciation can occur as the result of variation within the species which decreases the (...) probability of sharing genetic information. Critics of the Brooks-Wiley theory argue that they have abused terminology from information theory and t thermodynamics. In this paper I review the essentials of the theory, and give an account of hierarchical physical information systems within which the theory can be interpreted. I then show how the major conceptual objections can be answered. (shrink)

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about distinctions, differences and distinguishability and is formalized using the distinctions of a partition. All the definitions of simple, joint, conditional and mutual entropy of Shannon information theory are derived by a uniform transformation from the corresponding definitions at the logical level. The purpose of this (...) paper is to give the direct generalization to quantum logical information theory that similarly focuses on the pairs of eigenstates distinguished by an observable, i.e., qudits of an observable. The fundamental theorem for quantum logical entropy and measurement establishes a direct quantitative connection between the increase in quantum logical entropy due to a projective measurement and the eigenstates that are distinguished by the measurement. Both the classical and quantum versions of logical entropy have simple interpretations as “two-draw” probabilities for distinctions. The conclusion is that quantum logical entropy is the simple and natural notion of information for quantum information theory focusing on the distinguishing of quantum states. (shrink)

One well-known objection to the principle of maximum entropy is the so-called Judy Benjamin problem, first introduced by van Fraassen. The problem turns on the apparently puzzling fact that, on the basis of information relating an event’s conditional probability, the maximum entropy distribution will almost always assign to the event conditionalized on a probability strictly less than that assigned to it by the uniform distribution. In this article, I present an analysis of the Judy Benjamin problem that can (...) help to make sense of this seemingly odd feature of maximum entropy inference. My analysis is based on the claim that, in applying the principle of maximum entropy, Judy Benjamin is not acting out of a concern to maximize uncertainty in the face of new evidence, but is rather exercising a certain brand of epistemic charity towards her informant. This epistemic charity takes the form of an assumption on the part of Judy Benjamin that her informant’s evidential report leaves out no relevant information. Such a reconceptualization of the motives underlying Judy Benjamin’s appeal to the principle of maximum entropy can help to further our understanding of the true epistemological grounds of this principle and correct a common misapprehension regarding its relationship to the principle of insufficient reason. 1Introduction2The Principle of Maximum Entropy3An Apologia for Judy Benjamin4Conclusion: Entropy and Insufficient Reason. (shrink)

I discuss the statistical mechanics of gravitating systems and in particular its cosmological implications, and argue that many conventional views on this subject in the foundations of statistical mechanics embody significant confusion; I attempt to provide a clearer and more accurate account. In particular, I observe that (i) the role of gravity in entropy calculations must be distinguished from the entropy of gravity, that (ii) although gravitational collapse is entropy-increasing, this is not usually because the collapsing matter (...) itself increases in entropy, and that (iii) the Second Law of thermodynamics does not owe its validity to the statistical mechanics of gravitational collapse. (shrink)

Although the laws of thermodynamics are well established for black hole horizons, much less has been said in the literature to support the extension of these laws to more general settings such as an asymptotic de Sitter horizon or a Rindler horizon (the event horizon of an asymptotic uniformly accelerated observer). In the present paper we review the results that have been previously established and argue that the laws of black hole thermodynamics, as well as their underlying statistical mechanical content, (...) extend quite generally to what we call here “causal horizons.” The root of this generalization is the local notion of horizon entropy density. (shrink)

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an "a priori" probability that is uniform, and where all MAXENT constraints are limited to 0-1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 established a sensitivity (...) of MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events. (shrink)

I assess the thesis that counterfactual asymmetries are explained by an asymmetry of the global entropy at the temporal boundaries of the universe, by developing a method of evaluating counterfactuals that includes, as a background assumption, the low entropy of the early universe. The resulting theory attempts to vindicate the common practice of holding the past mostly fixed under counterfactual supposition while at the same time allowing the counterfactual's antecedent to obtain by a natural physical development. Although the (...) theory has some success in evaluating a wide variety of ordinary counterfactuals, it fails as an explanation of counterfactual asymmetry. (shrink)

"By combining recent advances in the physical sciences with some of the novel ideas, techniques, and data of modern biology, this book attempts to achieve a new and different kind of evolutionary synthesis. I found it to be challenging, fascinating, infuriating, and provocative, but certainly not dull."--James H, Brown, University of New Mexico "This book is unquestionably mandatory reading not only for every living biologist but for generations of biologists to come."--Jack P. Hailman, Animal Behaviour , review of the first (...) edition "An important contribution to modern evolutionary thinking. It fortifies the place of Evolutionary Theory among the other well-established natural laws."--R.Gessink, TAXON. (shrink)

Integrating concepts of maintenance and of origins is essential to explaining biological diversity. The unified theory of evolution attempts to find a common theme linking production rules inherent in biological systems, explaining the origin of biological order as a manifestation of the flow of energy and the flow of information on various spatial and temporal scales, with the recognition that natural selection is an evolutionarily relevant process. Biological systems persist in space and time by transfor ming energy from one state (...) to another in a manner that generates structures which allows the system to continue to persist. Two classes of energetic transformations allow this; heat-generating transformations, resulting in a net loss of energy from the system, and conservative transformations, changing unusable energy into states that can be stored and used subsequently. All conservative transformations in biological systems are coupled with heat-generating transformations; hence, inherent biological production, or genealogical proesses, is positively entropic. There is a self-organizing phenomenology common to genealogical phenomena, which imparts an arrow of time to biological systems. Natural selection, which by itself is time-reversible, contributes to the organization of the self-organized genealogical trajectories. The interplay of genealogical (diversity-promoting) and selective (diversity-limiting) processes produces biological order to which the primary contribution is genealogical history. Dynamic changes occuring on times scales shorter than speciation rates are microevolutionary; those occuring on time scales longer than speciation rates are macroevolutionary. Macroevolutionary processes are neither redicible to, nor autonomous from, microevolutionary processes. (shrink)

The idea that the changing entropy of a system is relevant to explaining why we know more about the system's past than about its future has been criticized on several fronts. This paper assesses the criticisms and clarifies the epistemology of the inference problem. It deploys a Markov process model to investigate the relationship between entropy and temporally asymmetric inference.

Views the process of artistic creation in light of the conflict between man's quest for order and increasing universal disorder.

Entropy and information are both emerging as currencies of interdisciplinary dialogue, most recently in evolutionary theory. If this dialogue is to be fruitful, there must be general agreement about the meaning of these terms. That this is not presently the case owes principally to the supposition of many information theorists that information theory has succeeded in generalizing the entropy concept. The present paper will consider the merits of the generalization thesis, and make some suggestions for restricting both (...) class='Hi'>entropy and information to specific arenas of discourse. (shrink)

In the sixth section of his light quantum paper of 1905, Einstein presented the miraculous argument, as I shall call it. Pointing out an analogy with ideal gases and dilute solutions, he showed that the macroscopic, thermodynamic properties of high frequency heat radiation carry a distinctive signature of finitely many, spatially localized, independent components and so inferred that it consists of quanta. I describe how Einstein’s other statistical papers of 1905 had already developed and exploited the idea that the ideal (...) gas law is another macroscopic signature of finitely many, spatially localized, independent components and that these papers in turn drew on his first two, “worthless” papers of 1901 and 1902 on intermolecular forces. However, while the ideal gas law was a secure signature of independence, it was harder to use as an indicator that there are finitely many components and that they are spatially localized. Further, since his analysis of the ideal gas law depended on the assumption that the number of components was fixed, its use was precluded for heat radiation, whose component quanta vary in number in most processes. So Einstein needed and found another, more powerful signature of discreteness applicable to heat radiation and which indicated all these properties. It used one of the few processes, volume fluctuation, in which heat radiation does not alter the number of quanta. (shrink)

We investigate uncertain reasoning with quantified sentences of the predicate calculus treated as the limiting case of maximum entropy inference applied to finite domains.

This essay is an attempt to reconcile the disturbing contradiction between the striving for order in nature and in man and the principle of entropy implicit in the second law of thermodynamics - between the tendency toward greater organization and the general trend of the material universe toward death and disorder.

Information, entropy, probability: these three terms are closely interconnected in the prevalent understanding of statistical mechanics, both when this field is taught to students at an introductory level and in advanced research into the field’s foundations. This paper examines the interconnection between these three notions in light of recent research in the foundations of statistical mechanics. It disentangles these concepts and highlights their differences, at the same time explaining why they came to be so closely linked in the literature. (...) In the literature the term ‘information’ is often linked to entropy and probability in discussions of Maxwell’s Demon and its attempted exorcism by the Landauer-Bennett thesis, and in analyses of the spin echo experiments. The direction taken in the present paper is a different one. Here we discuss the statistical mechanical underpinning of the notions of probability and entropy, and this constructive approach shows that information plays no fundamental role in these concepts, although it can be conveniently used in a sense that we shall specify. (shrink)

In the sixth section of his light quantum paper of 1905, Einstein presented the miraculous argument, as I shall call it. Pointing out an analogy with ideal gases and dilute solutions, he showed that the macroscopic, thermodynamic properties of high-frequency heat radiation carry a distinctive signature of finitely many, spatially localized, independent components and so inferred that it consists of quanta. I describe how Einstein's other statistical papers of 1905 had already developed and exploited the idea that the ideal gas (...) law is another macroscopic signature of finitely many, spatially localized, independent components and that these papers in turn drew on his first two, "worthless" papers of 1901 and 1902 on intermolecular forces. However, while the ideal gas law was a secure signature of independence, it was harder to use as an indicator that there are finitely many components and that they are spatially localized. Further, since his analysis of the ideal gas law depended on the assumption that the number of components was fixed, its use was precluded for heat radiation, whose component quanta vary in number in most processes. So Einstein needed and found another, more powerful signature of discreteness applicable to heat radiation and which indicated all these properties. It used one of the few processes, volume fluctuation, in which heat radiation does not alter the number of quanta. © 2005 Elsevier Ltd. All rights reserved. (shrink)

This essay is an attempt to reconcile the disturbing contradiction between the striving for order in nature and in man and the principle of entropy implicit in the second law of thermodynamics - between the tendency toward greater organization and the general trend of the material universe toward death and disorder.

Two open questions of inductive reasoning are solved: (1) does the principle of maximum entropy (pme) give a solution to the obverse Majerník problem; and (2) is Wagner correct when he claims that Jeffrey’s updating principle (jup) contradicts pme? Majerník shows that pme provides unique and plausible marginal probabilities, given conditional probabilities. The obverse problem posed here is whether pme also provides such conditional probabilities, given certain marginal probabilities. The theorem developed to solve the obverse Majerník problem demonstrates that (...) in the special case introduced by Wagner pme does not contradict jup, but elegantly generalizes it and offers a more integrated approach to probability updating. (shrink)

This paper argues that striving is a cardinal virtue in sport and life. It is an overlooked virtue that is an important component of human happiness and a source of a sense of dignity. The human ps...

In theories of gravity with a positive cosmological constant, we consider product solutions with flux, of the form (A)dS p ×S q . Most solutions are shown to be perturbatively unstable, including all uncharged dS p ×S q spacetimes. For dimensions greater than four, the stable class includes universes whose entropy exceeds that of de Sitter space, in violation of the conjectured “N-bound.” Hence, if quantum gravity theories with finite-dimensional Hilbert space exist, the specification of a positive cosmological constant (...) will not suffice to characterize the class of spacetimes they describe. (shrink)

Markov models of evolution describe changes in the probability distribution of the trait values a population might exhibit. In consequence, they also describe how entropy and conditional entropy values evolve, and how the mutual information that characterizes the relation between an earlier and a later moment in a lineage’s history depends on how much time separates them. These models therefore provide an interesting perspective on questions that usually are considered in the foundations of physics—when and why does (...) class='Hi'>entropy increase and at what rates do changes in entropy take place? They also throw light on an important epistemological question: are there limits on what your observations of the present can tell you about the evolutionary past? (shrink)

Traditional eschatology clashes with the theory of entropy. Trying to bridge the gap, Robert John Russell assumes that theology and science are based on contradictory, yet equally valid, metaphysical assumptions, each one capable of questioning and impacting the other. The author doubts that Russell's proposal will convince empirically oriented scientists and attempts to provide a viable alternative. Historical‐critical analysis suggests that biblical future expectations were redemptive responses to changing human needs. Apocalyptic visions were occasioned by heavy suffering in postexilic (...) times. Interpreted in realistic terms, they have since proved to be untenable. The expectation of a new creation without evil, suffering, and death is not constitutive for the substantive content of the biblical message as such. Biblical future expectations must be reconceptualized in terms of best contemporary insight and in line with a dynamic reading of the biblical witness as God's vision of comprehensive optimal well‐being that operates like a shifting horizon and opens up ever new vistas, challenges, and opportunities. (shrink)

The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the partition. The logical concept of entropy based on partition logic is the normalized counting measure of the set of distinctions of a partition on a finite set--just as the usual logical notion of probability based on the Boolean (...) logic of subsets is the normalized counting measure of the subsets (events). Thus logical entropy is a measure on the set of ordered pairs, and all the compound notions of entropy (join entropy, conditional entropy, and mutual information) arise in the usual way from the measure (e.g., the inclusion-exclusion principle)--just like the corresponding notions of probability. The usual Shannon entropy of a partition is developed by replacing the normalized count of distinctions (dits) by the average number of binary partitions (bits) necessary to make all the distinctions of the partition. (shrink)

It is shown that entropy increase in thermodynamic systems can plausibly be accounted for by the random action of vacuum radiation. A recent calculation by Rueda using stochastic electrodynamics (SED) shows that vacuum radiation causes a particle to undergo a rapid Brownian motion about its average dynamical trajectory. It is shown that the magnitude of spatial drift calculated by Rueda can also be predicted by assuming that the average magnitudes of random shifts in position and momentum of a particle (...) correspond to the lower limits of the uncertainty relation. The latter analysis yields a plausible expression for the shift in momentum caused by vacuum radiation. It is shown that when the latter shift in momentum is magnified in particle interactions, the fractional change in each momentum component is on the order of unity within a few collision times, for gases and (plausibly) for denser systems over a very broad range of physical conditions. So any system of particles in this broad range of conditions would move to maximum entropy, subject to its thermodynamic constraints, within a few collision times. It is shown that the spatial drift caused by vacuum radiation, as predicted by the above SED calculation, can be macroscopic in some circumstances, and an experimental test of this effect is proposed. Consistency of the above results with quantum mechanics is discussed, and it is shown that the diffusion constant associated with the above Brownian drift is the same as that used in stochastic interpretations of the Schrödinger equation. (shrink)

The language of entropy is examined for consistency with its mathematics and physics, and for its efficacy as a guide to what entropy means. Do common descriptors such as disorder, missing information, and multiplicity help or hinder understanding? Can the language of entropy be helpful in cases where entropy is not well defined? We argue in favor of the descriptor spreading, which entails space, time, and energy in a fundamental way. This includes spreading of energy spatially (...) during processes and temporal spreading over accessible microstates states in thermodynamic equilibrium. Various examples illustrate the value of the spreading metaphor. To provide further support for this metaphor’s utility, it is shown how a set of reasonable spreading properties can be used to derive the entropy function. A main conclusion is that it is appropriate to view entropy’s symbol S as shorthand for spreading. (shrink)

This paper strengthens and defends the pluralistic implications of Einstein's successful, quantitative predictions of Brownian motion for a philosophical dispute about the nature of scientific advance that began between two prominent philosophers of science in the second half of the twentieth century (Thomas Kuhn and Paul Feyerabend). Kuhn promoted a monistic phase-model of scientific advance, according to which a paradigm driven `normal science' gives rise to its own anomalies, which then lead to a crisis and eventually a scientific revolution. Feyerabend (...) stressed the importance of pluralism for scientific progress. He rejected Kuhn's model arguing that it fails to recognize the role that alternative theories can play in identifying exactly which phenomena are anomalous in the first place. On Feyerabend's account, Einstein's predictions allow for a crucial experiment between two incommensurable theories, and are an example of an anomaly that could refute the reigning paradigm only after the development of a competitor. Using Kuhn's specification of a disciplinary matrix to illustrate the incommensurability between the two paradigms, we examine the different research strategies available in this peculiar case. On the basis of our reconstruction, we conclude by rebutting some critics of Feyerabend's argument. (shrink)

The principle of maximum entropy is a general method to assign values to probability distributions on the basis of partial information. This principle, introduced by Jaynes in 1957, forms an extension of the classical principle of insufficient reason. It has been further generalized, both in mathematical formulation and in intended scope, into the principle of maximum relative entropy or of minimum information. It has been claimed that these principles are singled out as unique methods of statistical inference that (...) agree with certain compelling consistency requirements. This paper reviews these consistency arguments and the surrounding controversy. It is shown that the uniqueness proofs are flawed, or rest on unreasonably strong assumptions. A more general class of inference rules, maximizing the so-called Re[acute ]nyi entropies, is exhibited which also fulfill the reasonable part of the consistency assumptions. (shrink)

With reference to two specific modalities of sensation, the taste of saltiness of chloride salts, and the loudness of steady tones, it is shown that the laws of sensation (logarithmic and power laws) are expressions of the entropy per mole of the stimulus. That is, the laws of sensation are linear functions of molar entropy. In partial verification of this hypothesis, we are able to derive an approximate value for the gas constant, a fundamental physical constant, directly from (...) psychophysical measurements. The significance of our observation lies in the linking of the phenomenon of “sensation” directly to a physical measure. It suggests that if the laws of physics are universal, the laws of sensation and perception are similarly universal. It also connects the sensation of a simple, steady physical signal with the molecular structure of the signal: the greater the number of microstates or complexions of the stimulus signal, the greater the magnitude of the sensation (saltiness or loudness). The hypothesis is currently tested on two sensory modalities. (shrink)

I assess the thesis that counterfactual asymmetries are explained by an asymmetry of the global entropy at the temporal boundaries of the universe by developing a new semantic element for counterfactuals called objective assertibility and a method of evaluating counterfactuals that constrains consideration to possibilities where the early universe has low entropy. The resulting theory vindicates the common practice of holding the past mostly fixed under counterfactual supposition while at the same time allowing the counterfactual's antecedent to obtain (...) by a natural physical development. While excellent for evaluating a wide variety of ordinary counterfactuals, it does not fully succeed as an explanation of counterfactual asymmetry. (shrink)

According to the universal entropy bound, the entropy of a complete weakly self-gravitating physical system can be bounded exclusively in terms of its circumscribing radius and total gravitating energy. The bound’s correctness is supported by explicit statistical calculations of entropy, gedanken experiments involving the generalized second law, and Bousso’s covariant holographic bound. On the other hand, it is not always obvious in a particular example how the system avoids having too many states for given energy, and hence (...) violating the bound. We analyze in detail several purported counterexamples of this type, and exhibit in each case the mechanism behind the bound’s efficacy. (shrink)

Using computer simulations, we investigate the time evolution of the (Boltzmann) entropy of a dense fluid not in local equilibrium. The macrovariables M describing the system are the (empirical) particle density f = {f(x,v)} and the total energy E. We find that S(ft,E) is a monotone increasing in time even when its kinetic part is decreasing. We argue that for isolated Hamiltonian systems monotonicity of S(Mt) = S(MXt) should hold generally for ‘‘typical’’ (the overwhelming majority of) initial microstates (phase (...) points) X0 belonging to the initial macrostate M0, satisfying MX0 = M0. This is a consequence of Liouville's theorem when Mt evolves according to an autonomous deterministic law. (shrink)

I begin with the definition of power, and find that it is finalistic inasmuch as work directs energy dissipation in the interests of some system. The maximum power principle of Lotka and Odum implies an optimal energy efficiency for any work; optima are also finalities. I advance a statement of the maximum entropy production principle, suggesting that most work of dissipative structures is carried out at rates entailing energy flows faster than those that would associate with maximum power. This (...) is finalistic in the sense that the out-of-equilibrium universe, taken as an isolated system, entrains work in the interest of global thermodynamic equilibration. I posit an evolutionary scenario, with a development on Earth from abiotic times, when promoting convective energy flows could be viewed as the important function of dissipative structures, to biotic times when the preservation of living dissipative structures was added to the teleology. Dissipative structures are required by the equilibrating universe to enhance local energy gradient dissipation. (shrink)