Abstract
Inductive probabilistic reasoning is understood as the application of inference patterns that use statistical background information to assign (subjective) probabilities to single events. The simplest such inference pattern is direct inference: from “70% of As are Bs” and “a is an A” infer that a is a B with probability 0.7. Direct inference is generalized by Jeffrey’s rule and the principle of cross-entropy minimization. To adequately formalize inductive probabilistic reasoning is an interesting topic for artificial intelligence, as an autonomous system acting in a complex environment may have to base its actions on a probabilistic model of its environment, and the probabilities needed to form this model can often be obtained by combining statistical background information with particular observations made, i.e., by inductive probabilistic reasoning. In this paper a formal framework for inductive probabilistic reasoning is developed: syntactically it consists of an extension of the language of first-order predicate logic that allows to express statements about both statistical and subjective probabilities. Semantics for this representation language are developed that give rise to two distinct entailment relations: a relation ⊨ that models strict, probabilistically valid, inferences, and a relation that models inductive probabilistic inferences. The inductive entailment relation is obtained by implementing cross-entropy minimization in a preferred model semantics. A main objective of our approach is to ensure that for both entailment relations complete proof systems exist. This is achieved by allowing probability distributions in our semantic models that use non-standard probability values. A number of results are presented that show that in several important aspects the resulting logic behaves just like a logic based on real-valued probabilities alone.
Similar content being viewed by others
References
M. Abadi J.Y. Halpern (1994) ArticleTitleDecidability and expressiveness for first-order logics of probability Information and Computation 112 1–36 Occurrence Handle10.1006/inco.1994.1049
F. Bacchus (1990a) ArticleTitleLp, a logic for representing and reasoning with statistical knowledge Computational Intelligence 6 209–231
F. Bacchus (1990b) Representing and Reasoning With Probabilistic Knowledge MIT Press Cambridge
Bacchus F., Grove A., Halpern J., Koller D. (1992). From statistics to beliefs, In Proceedings of National Conference on Artificial Intelligence (AAAI-92)
F. Bacchus A.J. Grove J.Y. Halpern D. Koller (1997) ArticleTitleFrom statistical knowledge bases to degrees of belief Artificial Intelligence 87 75–143
Boole, G.: 1854, Investigations of Laws of Thought on which are Founded the Mathematical Theories of Logic and Probabilities, London.
R. Carnap (1950) Logical Foundations of Probability The University of Chicago Press Chicago
R. Carnap (1952) The Continuum of Inductive Methods The University of Chicago Press Chicago
B.I. Dahn H. Wolter (1983) ArticleTitleOn the theory of exponential fields Zeitschrift fürmathematische Logik und Grundlagen der Mathematik 29 465–480
de Finetti B. (1937). La prévision: ses lois logiques, ses sources subjectives. Annales de l’Institut Henri Poincaré. English Translation in (Kyburg and Smokler 1964)
A.P. Dempster (1967) ArticleTitleUpper and lower probabilities induced by a multivalued mapping Annals of Mathematical Statistics 38 325–339
P. Diaconis S. Zabell (1982) ArticleTitleUpdating subjective probability Journal of the American Statistical Association 77 IssueID380 822–830
Dubois, D. and H. Prade: 1997, Focusing vs. belief revision: A fundamental distinction when dealing with generic knowledge, In Proceedings of the First International Joint Conference on Qualitative and Quantitative Practical Reasoning, Springer, pp. 96--107.
J.E. Fenstad (1967) Representations of probabilities defined on first order languages J.N. Crossley (Eds) Sets, Models and Recursion Theory Amsterdam North Holland 156–172
Gaifman H. (1964). Concerning measures in first order calculi. Israel Journal of Mathematics 2
H. Gaifman M. Snir (1982) ArticleTitleProbabilities over rich languages, testing and randomness Journal of Symbolic Logic 47 IssueID3 495–548
I. Gilboa D. Schmeidler (1993) ArticleTitleUpdatin ambiguous beliefs Journal of Economic Theory 59 33–49 Occurrence Handle10.1006/jeth.1993.1003
Grove A., Halpern J. (1998). Updating sets of probabilities. In Proceedings of the 14th Conference on Uncertainty in AI. pp. 173–182
Grove A., Halpern J., Koller D. (1992a). Asymptotic conditional probabilities for first-order logic. In Proceedings of the 24th ACM Symposium on Theory of Computing
Grove A., Halpern J., Koller D. (1992b). Random worlds and maximum entropy. In Proceedings of the 7th IEEE Symposium on Logic in Computer Science
T. Hailperin (1976) Boole’s Logic and Probability Vol. 85 of Studies in Logic and the Foundations of Mathematics Amsterdam North Holland
T. Hailperin (1996) Sentential Probability Logic Lehigh University Press Bethlehem
J. Halpern (1990) ArticleTitleAn analysis of first-order logics of probability Artificial Intelligence 46 311–350 Occurrence Handle10.1016/0004-3702(90)90019-V
D.N. Hoover (1978) ArticleTitleProbability logic Annals of Mathematical Logic 14 287–313 Occurrence Handle10.1016/0003-4843(78)90022-0
M. Jaeger (1994) A logic for default reasoning about probabilities R. Lopez de Mantaraz D. Poole (Eds) Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence (UAI’94) Morgan Kaufmann Seattle, USA 352–359
M. Jaeger (1994) Probabilistic reasoning in terminological logics J. Doyle E. Sandewall P. Torasso (Eds) Principles of Knowledge Representation an Reasoning: Proceedings of the 4th International Conference (KR94) Morgan Kaufmann Bonn, Germany 305–316
Jaeger M. (1995a). Default Reasoning about Probabilities. PhD thesis. Universität des Saarlandes
M. Jaeger (1995) Minimum cross-entropy reasoning: A statistical justification C.S. Mellish (Eds) Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95). Morgan Kaufmann Montréal, Canada 1847–1852
E. Jaynes (1978) Where do we stand on maximum entropy? R. Levine M. Tribus (Eds) The Maximum Entropy Formalism MIT Press Cambridge 15–118
R. Jeffrey (1965) The Logic of Decision McGraw-Hill New York
F. Jensen (2001) Bayesian Networks and Decision Graphs Springer Berlin
Keisler, H.: 1985, Probability quantifiers, In J. Barwise and S. Feferman (eds.), Model-Theoretic Logics, Springer, pp. 509--556.
S. Kullback (1959) Information Theory and Statistics Wiley New York
S. Kullback R.A. Leibler (1951) ArticleTitleOn information and sufficiency Annals of mathematical statistics 22 79–86
Kyburg, H.E.: 1974,The Logical Foundations of Statistical Inference, D. Reidel Publishing company.
H.E. Kyburg (1983) ArticleTitleThe reference class Philosophy of Science 50 374–397
H.E. Kyburg H.E. Smokler (Eds) (1964) Studies in Subjective Probability Wiley New York
D. Lewis (1976) ArticleTitleProbabilities of conditionals and conditional probabilities The Philosophical Review 85 IssueID3 297–315
J. McCarthy (1980) ArticleTitleCircumscription – a form of non-monotonic reasoning Artificial Intelligence 13 27–39
S. Moral N. Wilson (1995) Revision rules for convex sets of probabilities G. Coletti D. Dubois R. Scozzafava (Eds) Mathematical Models for Handling Partial Knowledge in Artificial Intelligence Kluwer Dordrecht
N. Nilsson (1986) ArticleTitleProbabilistic logic Artificial Intelligence 28 71–88 Occurrence Handle10.1016/0004-3702(86)90031-7
J. Paris A. Vencovská (1990) ArticleTitleA note on the inevitability of maximum entropy International Journal of Approximate Reasoning 4 183–223 Occurrence Handle10.1016/0888-613X(90)90020-3
J. Paris A. Vencovská (1992) ArticleTitleA method for updating that justifies minimum cross entropy International Journal of Approximate Reasoning 7 1–18 Occurrence Handle10.1016/0888-613X(92)90022-R
J.B. Paris (1994) The Uncertain Reasoner’s Companion Cambridge University Press Cambridge
J. Pearl (1988) Probabilistic Reasoning in Intelligent Systems : Networks of Plausible Inference The Morgan Kaufmann series in representation and reasoning rev. 2nd pr. edn. Morgan Kaufmann San Mateo, CA
J.L. Pollock (1983) ArticleTitleA theory of direct inference Theory and Decision 15 29–95 Occurrence Handle10.1007/BF00133461
M.O. Rabin (1977) Decidable theories J. Barwise (Eds) Handbook of mathematical logic Elsevier Science Publishers Amsterdam
H. Reichenbach (1949) The Theory of Probability University of California Press Berkely, CA
L.J. Savage (1954) The Foundations of Statistics Wiley New York
D. Scott P. Krauss (1966) Assigning probabilities to logical formulas J. Hintikka P. Suppes (Eds) Aspects of Inductive Logic Amsterdam North Holland 219–264
G. Shafer (1976) A Mathematical Theory of Evidence Princeton University Press New Jersey
Shoham Y. (1987). Nonmonotonic logics: Meaning and utility. In Proceedings of IJCAI-87
J. Shore R. Johnson (1980) ArticleTitleAxiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy IEEE Transactions on Information Theory IT-26 IssueID1 26–37
J. Shore R. Johnson (1983) ArticleTitleComments on and correction to “Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy” IEEE Transactions on Information Theory IT-29 IssueID6 942–943
von Mises, R.: 1951, Wahrscheinlichkeit Statisik und Wahrheit, Springer.
R. Mises Particlevon (1957) Probability Statistics and Truth George Allen & Unwin London
P. Walley (1991) Statistical Reasoning with Imprecise Probabilities Chapman & Hall London
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jaeger, M. A Logic For Inductive Probabilistic Reasoning. Synthese 144, 181–248 (2005). https://doi.org/10.1007/s11229-004-6153-2
Issue Date:
DOI: https://doi.org/10.1007/s11229-004-6153-2