Can someone explain to me how to make sense of direct realism, more precisely : how can one claim that to perceive is to have direct access to the object itself if we grant that perceiving is the end product of a certain pattern of neurons firing ?

I can understand direct realism on aristotelian grounds where an objective form leaves the object and penetrates the intellect, but if firing neurons are involved, aren't we obliged to say that the brain reconstructs the "thing in itself" ? (I understand also the problems involved with the theory of sense-data and the motivations that originate from physicalism : my question is purely regarding the constraints imposed by basic neurological ideas).