One way to make sense of direct realism is to place its claims in the context of human life as we ordinarily understand it, and see what they entail.
If all participants in this forum were attending a public lecture, and the speaker held up his hand for silence, we can all agree that, as (sighted) persons about to listen to a talk, we in fact were witness to a shared event, namely the speaker's hand-raising.  This would be evidenced by our responding appropriately and settling down.  This fact remains true whatever our individual visual experiences as witnesses, and whatever physical micro events occur.   This is all direct realism really seems to amount to: that as persons sharing a world with each other we in fact witness and partake in the same events and circumstances.  

Perhaps "shared realism" would be a less misleading term; no noumena required!