Abstract
Objective Bayesians hold that degrees of belief ought to be chosen in the set of probability functions calibrated with one’s evidence. The particular choice of degrees of belief is via some objective, i.e., not agent-dependent, inference process that, in general, selects the most equivocal probabilities from among those compatible with one’s evidence. Maximising entropy is what drives these inference processes in recent works by Williamson and Masterton though they disagree as to what should have its entropy maximised. With regard to the probability function one should adopt as one’s belief function, Williamson advocates selecting the probability function with greatest entropy compatible with one’s evidence while Masterton advocates selecting the expected probability function relative to the density function with greatest entropy compatible with one’s evidence. In this paper we discuss the significant relative strengths of these two positions. In particular, Masterton’s original proposal is further developed and investigated to reveal its significant properties; including its equivalence to the centre of mass inference process and its ability to accommodate higher order evidence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In awkward cases “maximally equivocal” is replaced by “sufficiently equivocal”, but for the most part we shall ignore this subtlety.
To avoid unnecessary complications we here work over a propositional language; rather than a language of first-order logic.
Both ourselves and Williamson freely allow that there may be other types of evidence and that these will place their own constraints on reasonable credence. Hence, we allow that the calibration norm as explicated herein may be incomplete. This does not make what is presented here unsound, it merely means that it may not be the full story on calibration.
Nothing important hinges on this, we are happy with every approach to evidence as long as it results in a set of calibrated functions \({\mathbb {P}}^*\).
Two different densities \(\varrho ,\varrho '\in \mathbb {C}_{{\mathbb {P}}^*}^1\) which only differ on a null-set of \({\mathbb {P}}^*\) have the same entropy. However, the expected probabilities with respect to \(\varrho ,\varrho '\) are equal: i.e., \(\int _{P\in {\mathbb {P}}^*}\varrho (P)\cdot PdP=\int _{P\in {\mathbb {P}}^*}\varrho '(P)\cdot PdP\). Null-sets of \({\mathbb {P}}^*\) are thus of no interest to us. With some abuse of language we say that the calibrated density with greatest entropy \(\varrho ^\dagger \) equals the uniform density on \({\mathbb {P}}^*\). We do this in spite of the fact that there exist densities with the same entropy as \(\varrho ^\dagger \).
Williamson’s (2010, pp. 64–65) argument here is as follows: If an agent’s probability function P is not in the convex hull of the \({\mathbb {P}}^*\) determined by their evidence, then there is some other probability function \(P'\in \langle {\mathbb {P}}^*\rangle \) which has a strictly better worst case expected logarithmic loss than P as shown by Grünwald and Dawid (2004); see also Landes (2015); Landes and Williamson 2013).
This problem is best exemplified by considering a sentence \(\theta \) that, according to our evidence, is settled one way or the other; so that \({\mathbb {P}}^*=\{P:P(\theta )\in \{0,1\}\}\). Arguably, restricting credence to any subset of \(\langle {\mathbb {P}}^*\rangle \) other than \({\mathbb {P}}^*\) would be arbitrary, but restricting credence to \({\mathbb {P}}^*\) would, by Williamson’s equivocation norm, yield the conclusion that one should either be certain of \(\theta \) or else be certain of its negation. This is highly counterintuitive, as typically one would think that in such a situation one should be as certain in the sentence as its negation, which is the result one obtains by applying the equivocation norm to \(\langle {\mathbb {P}}^*\rangle \). Thus no, non-arbitrary, subset of \(\langle {\mathbb {P}}^*\rangle \) avoids the issues posed by disjunctive evidence, while \(\langle {\mathbb {P}}^*\rangle \) does avoid those issues.
Degrees of belief in tautologies in \({\mathcal {L}}\) are one and degrees of belief in contradictions are zero; which is consistent with the axioms of probability.
While Masterton and Williamson avoid the unique probability objection raised by Bandyopadhyay and Brittan (2010), nothing in their previous writings deals with Bandyopadhyay’s et al criticism that often in science hypotheses are accepted on the strength of the evidence despite their posterior probability in the light of such evidence being very low. While Masterton and Williamson do not have posterior probabilities in their accounts, it is the case that significant evidence for a hypothesis may fail to result in high credence in that hypothesis in their frameworks. Masterton’s response to this concern is that while a high degree of belief in a hypothesis is a sufficient condition for its acceptance, it is not a necessary one. Thus, he allows that a hypothesis’ acceptance by an agent may be warranted by, e.g., a significant experimental result even when the reasonable degree of belief for that agent on the basis of that evidence is low. That is Masterton, much like (van Fraassen 1980), holds acceptance and belief to be two entirely distinct doxastic states where warranted acceptance is easier to come by than warranted belief.
To keep the example simple, we restrict ourselves here to the case in which there is no information as to how likely any given situation is. If such information is available, then it ought to be taken into account, in an appropriate manner. For example, if situation i is taken to be twice as likely as situation k, then situation i should be given double the weight of situation k. We shall come back to higher order evidence in Sect. 4. Lewis describes a similar procedure at Lewis (1980, p. 266).
Another approach satisfying language invariance, via marginalisation of Dirichlet priors, has been taken in Lawry and Wilmers (1994).
If \(\omega \in \Omega \setminus \Omega _I\), then \(P(\omega )=0\) for all \(P\in {\mathbb {P}}^*\). Hence, \(\log (P(\omega ))=-\infty \). So, would the above sum contain such an \(\omega \), then the entire sum would have value \(-\infty \).
Interior and exterior are here understood in the induced topology on \({\mathbb {P}}^*\). If \({\mathbb {P}}^*\) has a lower dimension than \({\mathbb {P}}^*\), then \(P^\dagger _{CM}\) is not necessarily an interior point of \({\mathbb {P}}^*\).
The same holds mutatis mutandis for \(P^\dagger _{CM_\infty }\).
Non-convex \({\mathbb {P}}^*\) are notoriously hard cases. The study of note in this case which does not simply consider the convex hull of \({\mathbb {P}}^*\) is Paris and Vencovská (2001).
Paris & Vensovská have, of course, also found a natural way in which Maxent is uniquely rational for the purposes of estimating objective probabilities (Paris and Vencovská 1989). In a later paper (Paris 2005, pp. 275–276), Jeff Paris is somewhat more sympathetic towards CM for estimating objective probabilities.
References
Adamčík, M. (2014). The information geometry of bregman divergences and some applications in multi-expert reasoning. Entropy, 16(12), 6338–6381.
Bandyopadhyay, P. S., & Brittan, G. J. (2010). Two dogmas of strong objective bayesianism. International Studies in the Philosophy of Science, 24(1), 45–65.
Bernardo, J. M., & Ramón, (1998). An introduction to bayesian reference analysis: Inference on the ratio of multinomial parameters. The Statistician, 47(1), 101–135.
Carnap, R. (1950). Logical foundations of probability. Chicago: University of Chicago Press.
de Finetti, B. (1989). Probabilism: A critical essay on the theory of probability and on the value of science. Erkenntnis, 31, 169–223.
Gaifman, H. (1986). A theory of higher order probabilities. In Proceedings of the Conference on Theoretical Aspects of Reasoning about Knowledge, pp. 275–292, California. Monterey.
Grünwald, P. D., & Dawid, A. P. (2004). Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. Annals of Statistics, 32(4), 1367–1433.
Halpern, J. Y., & Koller, D. (2004). Representation dependence in probabilistic inference. Journal of Artificial Intelligence Research, 21, 319–356.
Jaynes, E. T. (1968). Prior probabilities. IEEE Transactions On Systems Science and Cybernetics, 4(3), 227–241.
Jaynes, E. T. (2003). Probability theory: The logic of science. Cambridge: Cambridge University Press.
Joyce, J. M. (2010). A defense of imprecise credences in inference and decision making. Philosophical perspectives, 24(1), 281–323.
Landes, J. (2015). Probabilism, entropies and strictly proper scoring rules. International Journal of Approximate Reasoning, 63, 1–21.
Landes, J., & Williamson, J. (2013). Objective Bayesianism and the maximum entropy principle. Entropy, 15(9), 3528–3591.
Lawry, J., & Wilmers, G. M. (1994). An axiomatic approach to systems of prior distributions in inexact reasoning. In M. Masuch, & L. Pólos (Eds), Knowledge Representation and Reasoning Under Uncertainty, volume 808 of LNCS, pages 81–89. Springer.
Lewis, D. (1980). A subjectivist’s guide to objective chance. In R. C. Jeffrey (Ed), Studies in Inductive Logic and Probability, volume 2, chapter 13, pp. 263–293. Berkeley University Press.
Masterton, G. (2015). Equivocation for the objective Bayesian. Erkenntnis, 80, 403–432.
Miller, D. (1966). A paradox of information. British Journal for the Philosophy of Science, 17(1), 59–61.
Paris, J. B. (1994). The uncertain reasoner’s companion. Cambridge: Cambridge Univerity Press.
Paris, J. B. (2005). On filling-in missing conditional probabilities in causal networks. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 13(3), 263–280.
Paris, J. B. (2014). What you see is what you get. Entropy, 16(11), 6186–6194.
Paris, J. B., & Vencovská, A. (1989). On the applicability of maximum entropy to inexact reasoning. International Journal of Approximate Reasoning, 3(1), 1–34.
Paris, J. B., & Vencovská, A. (1990). A note on the inevitability of maximum entropy. International Journal of Approximate Reasoning, 4(3), 183–223.
Paris, J. B., & Vencovská, A. (1992). A method for updating that justifies minimum cross entropy. International Journal of Approximate Reasoning, 7(1–2), 1–18.
Paris, J. B., & Vencovská, A. (1997). In defense of the maximum entropy inference process. International Journal of Approximate Reasoning, 17(1), 77–103.
Paris, J. B., & Vencovská, A. (2001). Common sense and stochastic independence. In D. Corfield & J. Williamson (Eds.), Foundations of bayesianism (pp. 203–240). Dordrecht: Kluwer.
van Fraassen, B. (1980). The scientific image. Oxford: Clarendon Press.
Williamson, J. (2010). In defence of objective bayesianism. Oxford: Oxford University Press.
Williamson, J. (2014). How uncertain do we need to be? Erkenntnis, 79(6), 1249–1271.
Wilmers, G. (2015). A foundational approach to generalising the maximum entropy inference process to the multi-agent context. Entropy, 17(2), 594–645.
Acknowledgments
The authors are indebted to Erik J Olsson, Soroush Rafiee Rad and Jon Williamson for helpful comments and discussions.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
The main part of the work was carried out while Jürgen Landes was at Kent where he was supported by an UK Arts and Humanities Research Council funded project: From objective Bayesian epistemology to inductive logic. The final touches were put on while he was in Munich supported by the European Research Council grant 639276.
Rights and permissions
About this article
Cite this article
Landes, J., Masterton, G. Invariant Equivocation. Erkenn 82, 141–167 (2017). https://doi.org/10.1007/s10670-016-9810-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10670-016-9810-1