When I say that "Dungeons & Dragons was a great big, gloriously incoherent mess," as I did the other day, here's my primary point: D&D isn't really about anything, except perhaps fantasy adventures, with "fantasy" being defined so broadly as to include, literally, anything that isn't possible in the real world and "adventure" being defined almost as broadly. D&D doesn't devote much time to telling you what it's about. Its rulebooks are never self-conscious enough to devote any pages to the game's "themes," for example, and its explicit literary allusions consist primarily of bibliographies of inspirational books rather than anything more concrete. Consequently, the question of what D&D is about falls to each player and each referee to answer, with there being no single answer that is "right" or "wrong," even if some answers might be closer to what Gygax or Arneson might have had in mind.
This isn't to deny that Dungeons & Dragons includes lots of "markers" that stake out its territory. Its selection of character classes and races and the presence of magic and monsters all imply a world largely derived from Western ancient and medieval legend, supplemented with more recent fantasies deriving from the same. At the same time, few of the game's markers are so well-defined as to exclude the importation of ideas and concepts from other sources, thereby making it easy, say, to drop robots or dimension-hopping travelers into your campaign without undermining its essential "D&D-ness." Indeed, I'd argue that this kitchen sink approach is one of the most vital parts of what makes D&D the game it is.
Secondarily, my point is that the rules of Dungeons & Dragons weren't created according to some a priori plan. Rather, they grew organically through play to meet very specific demands. D&D is thus a Frankenstein's monster and, while many may be repulsed by the stitching and bolts that hold this patchwork creature together, I continue to marvel at the fact that it came to life in the first place. Of course, because the game's rules have this quality to them, they readily admit to the bolting on of yet more rules without much worry that doing so will somehow undermine its designers' "vision." I'd be hard pressed to believe that either Dave or Gary had much of a vision for their game beyond creating and sharing something that they found enjoyable to play.
The RPGs I enjoy most are those that are, to varying degrees, great big messes. I like rough edges in my games, things that rub me the wrong way and thereby encourage me to engage the game actively rather than passively. A terrific side effect of a game's being messy is that such games rarely tell you what you should do with them. That's probably why, of late, I've been so engrossed in games like Space Opera, another great big mess of a game that, even while it's providing stats for nearly every weapon conceived by man from the dawn of history to the far future, never once stops to tell you what to do with all these rules systems and sub-systems it so generously provides. I like that. Heck, I love that and wouldn't mind a few more games in this vein.