Simplicity tracks truth because compression tracks probability
Abstract
The simplicity of a theory seems closely related to how well the theory summarizes individual data points. Think, for example, of classic curve-fitting. It is easy to get perfect data-fit with a ‘‘theory’’ that simply lists each point of data, but such a theory is maximally unsimple (for the data-fit). The simple theory suggests instead that there is one underlying curve that summarizes this data, and we usually prefer such a theory even at some expense in data-fit. In general, it seems, theorizing involves looking for regularities or patterns in our experience, and such regularities are interesting to us because they summarize how our experience goes. We could list all the ravens we’ve encountered, and their colors, or we could summarize..