Penelope Maddy's original solution to the dilemma posed by Benacerraf in his 'Mathematical Truth' was to reconcile mathematical empiricism with mathematical realism by arguing that we can perceive realistically construed sets. Though her hypothesis has attracted considerable critical attention, much of it, in my view, misses the point. In this paper I vigorously defend Maddy's account against published criticisms, not because I think it is true, but because these criticisms have functioned to obscure a more fundamental issue that is well (...) worth addressing: in general -- and not only in the mathematical domain -- empiricism and realism simply cannot be reconciled by means of an account of perception anything like Maddy's. But because Maddy's account of perception is so plausible, this conclusion raises the specter of the broader incompatibility of realism and empiricism, which contemporary philosophers are frequently at pains to forget. (shrink)