Should magic in fantasy be logical, rule-governed?

As an aspiring fantasy writer, I’ve read and listened to lots of advice on writing in general, and on SF genre writing in particular. One of the axioms repeated by many fantasy writers is that, even though magic is one of the genre’s defining characteristics, it should be presented in a logical, rule-governed manner.

I’m not sure this is so. Recently I’ve been rereading Lord of the Rings, as I do once or twice a year like clockwork. Gandalf’s magic, and the magic practiced by the book’s other Powers, is never fully explained. It’s just there. I thinks this works to Tolkien’s benefit in a couple of ways. First, mystery comprises one of the epic’s major themes. In this respect Tolkien captures the essence of John Keats’s negative capability.

I had not a dispute but a disquisition, with Dilke on various subjects; several things dove-tailed in my mind, and at once it struck me what quality went to form a Man of Achievement, especially in Literature, and which Shakespeare possessed so enormously – I mean Negative Capability, that is, when a man is capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact and reason-Coleridge, for instance, would let go by a fine isolated verisimilitude caught from the Penetralium of mystery, from being incapable of remaining content with half-knowledge. This pursued through volumes would perhaps take us no further than this, that with a great poet the sense of Beauty overcomes every other consideration, or rather obliterates all consideration.

The countless allusions and references to people, places, and events in Middle Earth’s past which are never explained within the text force the reader to remain in a constant, low-level state of uncertainty. A prime example of this is the idiomatic reference when Aragorn says of Gandalf, “He is surer of finding the way home in a blind night than the cats of Queen Berúthiel.” Though Tolkien expounds on this figure elsewhere, the reader of Lord of the Rings never learns who she is, and hence she becomes part of the richness and depth of Tolkien’s world. In a similar way, Gandalf’s magic suggests whole realms of unexplored lore.

Secondly, though Gandalf’s magic is never fully explained, it is clearly finite. In perhaps the closest Tolkien comes to developing the workings of the wizard’s craft, Gandalf grumbles, while the Fellowship toils through the Pass of Caradhras, that “I cannot burn snow.” Tolkien rarely resorts to employing Gandalf’s magic as a deus ex machina device.

Having said that, I’ve also enjoyed fantasy authors who systematically approach magic in their novels. I found David Eddings’s construction of “the Will and the Word” to be one of the more enjoyable aspects of his Belgariad series, and of course the symbolic, sympathetic magic of the Harry Potter series, recalling Plato’s definition of forms and the “as above, so below” approach lends itself to hours of erudite exploration, even though the books themselves are at times juvenile.

I suppose my conclusion is that the extent to which a writer chooses to develop or not to develop a world’s magic system largely depends on the novel’s style and tone. In whimsical fantasy like Tolkien’s Lord of the Rings and Patricia A. McKillip’s Od Magic, to over-analyze the magic would actually detract from the world rather than add to it. But in other fantasy sub-genres, which in extreme cases might almost be called “science fantasy,” a sound logical foundation for the book’s magical workings are mandatory. The crime novel style of Jim Butcher’s Dresden Files demands that wizard Harry Dresden employ his powers of deduction in reasoning out the magical mechanics behind supernatural misdeeds.

But enough from me. I’m curious to know what you think. Discuss in the comments.