I'd argue the label "direct realist" is a red herring -- a kind of strawman label, misapplied to those who defend the honor of an ordinary, vulgar, "naive," commonsense realism.   (Armstrong, Strawson, Ayer, and Fumerton all seem, to me, to have been guilty of this on various occasions.)   

First of all, because it's nearly impossible to divide philosophers according to whether or not they think we can perceive tables and chairs "directly" (or "immediately") on the one hand, versus perceiving them "indirectly" (or "mediately") on the other. Are we supposed to imagine, for example, that "direct" realists deny the obvious intermediary role played by eyeballs and/or light in order to see tables and chairs? In other words, just how indirect does perception have to be, to be "indirect"?  


Do you have to go so far as to posit a full-blown chair in between yourself and an invisible, mind-independent we-know-not-what, in order to get yourself placed on the "indirect" side of the ledger? Or are mere light rays or optic nerves medium enough for mediacy? If a distant star viewed through a telescope is described as being perceived indirectly, is that enough to make the perception of an apple indirect? What if you grab the apple and take a bite out of it? What if somebody throws the apple at you, and it hits you so hard it knocks you over? What if you get hit by a bus? What if a brain surgeon pokes you in the brain with a pencil? Obviously, directness is, in this sense, a matter of degree.
 
The thing is, it's easy to agree that a complex process is involved in sense perception, including, for example, in the case of vision, a role for light rays, retinas, lenses, rods, cones, optic nerves, and the visual cortex, at least, to say nothing of any other neural processes. And, of course, a complex situation can be divided into a number of pieces, if not completely arbitrarily, at least in more than one way. But this fact makes the question of the relative "directness" of sense perception a completely trivial one. Besides, realists and anti-realists have never really quarreled over any particular account of corneas, retinas, optic nerves, or light rays. So, viewing the realism debate like this, as if it were a dispute over how relatively "direct" the biological process of perception is, can't be the right way to understand the issue, because people who don't disagree about the process can't truly be having a very meaningful dispute about how relatively direct it is, since, when it comes to the question of how relatively direct perception is, they basically agree. So, how could this really be the issue?
 
But the clincher here is the fact that there could hardly be a greater fan of the immediacy and directness of the visual perception of chairs than the definitive idealist, George Berkeley, yet, surely, it must be hard for anyone to get further than Berkeley from either realism or naivete. 


Indeed, Kant likewise insisted that our experience of tables and chairs is "immediate," especially since, for Kant, this "immediacy" was one of the main attractions of idealism.


The question isn't whether or not you can perceive tables and chairs "directly," or even how directly, more or less, you can, or can't, perceive them. The question is whether or not that chair you're sitting on right now, however more or less "directly" or "immediately" you suppose you perceive it, is, in your opinion, a ready-made thing in itself existing, persisting and subsisting independently, outside of thought and apart from the mind. In other words, the question is whether you consider that chair, this screen you're reading, and the ground beneath your feet, to be things in themselves which would probably not be affected too severely if every mind in the universe were suddenly annihilated.
 
The deeper controversy isn't about the relative directness of perception. It's a dispute about things like the mind-independence, substantiality, knowability, color, subsistence, meaning, reference, priority, objectivity, relativity, mentality, materiality, publicity, externality, identity, subjectivity and social construction of your chair. It's about questions like whether you are in the room, or the room is in you. It's about whether and how two different persons can perceive the same chair. It's about what it might mean to say that gold could have existed in the world without any minds to notice that gold, or name it. 


A fuller explanation is here:


http://queenelson.blogspot.com/2008/08/what-realism-can-be.html