From PhilPapers forum Discussion of David Bourget & David J. Chalmers, What do philosophers believe?:
Surprise: a meta-comment on the metasurvey
University of New Haven
This may be a commonplace in statistical science, but it came as a pleasant surprise to me to see "It is surprising" operationally defined in this paper, namely as reaching a level of dashed expectation by philosophers who took the metasurvey. "It is surprising" is one of countless expressions that, in my view, are used to subtly and illicitly but powerfully and even unawares used to bring others around to seeing things the way oneself does. I described an example in this passage:
The point I want to make in the present chapter is that the natural tendency to objectify what is essentially subjective is pervasive in our experience, even beyond morality. Consider the seemingly innocuous sentence, “The results were surprising,” which I quote from a book about the physiology and psychology of marine animals. The context is the discussion of an experiment to determine whether crustaceans can feel pain. The subjects were hermit crabs living inside abandoned snail shells that had been outfitted by the experimenters with electric probes. The “surprise” was that, in order to escape “low-voltage” shocks administered from the probes, some of the crabs left the protection of the shells they normally inhabit to elude predators, thereby putting themselves at much greater risk.
But to me it was not surprising at all that the crabs might find even “low-voltage” electric shocks aversive enough to induce them to do something foolish. Indeed, is not the entire scenario riddled with human subjectivity? For example, the voltage is presumably “low” in human terms; but why assume it would have this designation for crabs (which we now clearly see it does not)? To me it is unsurprising that some of the crabs might be, as it were, jumping into the fire, because they are jumping from a frying pan (in this case, an electrified shell). So only someone bringing certain assumptions to the experiment would be surprised. (Just as someone else, such as myself, bringing different assumptions would not be.) That in itself is a commonplace. My objection, then, is to the manner of expressing this response, as if it were an objective fact … just as if one were to say, “The crabs left the protection of their shells,” an objective fact, or even, “Some may find this surprising,” another fact. But to say that “the results were surprising” tout court is simply unwarranted. (It’s Just a Feeling, 2013, pp. 58-59)
Of course this is just the sort of biased "intuition" that experimental philosophers are forever urging us to be alert to.