Writer and editor Cheryl Morgan has a fine post up at Salon Futura analyzing the notion of 'genre', something I've dealt with here in the past: "What is Genre Anyway?"
Genre is less a self-imposed writerly straightjacket than it is an outside-imposed marketing bin in which to toss works that are in some way similar. It's a form of branding that allows potential readers to know what kind of thing they're likely to find when they open the pages. (This is a notion that Morgan finds in William Gibson and Gary Wolfe).
Interestingly, this week's SF Signal podcast deals with a similar issue: "Can You Judge a Book By Its Cover?" The participants note that many books which fit comfortably into the SF genre are sometimes given non-typical cover art if the publisher believes that they might have crossover appeal.
Presumably we're in this situation because there is a certain segment of the public that would really like some of the material that falls under SF, but that they would be otherwise unwilling to read it because they have an aversion to the genre itself due to a misunderstanding of what one finds in it.
Morgan also notes, correctly I think, that though Ed Docx dismissed 'genre' as "a constrained form of writing", it's firstly not at all clear that constraint is a bad thing (think of a sonnet, constrained if anything is), and secondly if SF is constrained, it's not at all clear what that constraint amounts to. Perhaps SF is a form of literature that muses about scientific topics in some sense, or about some imagined scientific or technological change. But if so the 'constraint' is almost absurdly thin.
More's the point, when a writer looks to begin a new work, I submit that she does not typically look first to the constraints to see what cannot be written, but rather to what she is interested in writing. If it's a piece about some new scientific discovery, then her publisher will brand it SF, unless of course they take it to be of more general interest in which case they may well simply call it "fiction", as is my copy of Charles Yu's How to Live Safely in a Science Fictional Universe, as plainly a work of SF as anything I've seen.
Are there formulaic writers of SF? Certainly, just as there are formulaic writers, artists, producers of every kind. And even the best have been known to fall into their own ruts of formula from time to time. Are all of Shakespeare's plays or Haydn's concertos really of the same quality?
My supposition with people like Docx and the general disdain for genre in certain literary circles is that "literary" is often used simply as a term of praise. That is, a work labeled 'genre' or 'SF' is taken to be labeled as merely genre or merely SF. And then, presumably, fiction that is in the realist or (forgive me) "literary" genre is simply termed fiction unless it is good enough to qualify as literature.
That's fine so far as it goes, but it's liable to confusion. It gives us two rather conflicting uses of the term "literary". One picks out a genre of modern, generally realist fiction, interested in smaller focus stories of contemporary life. And another picks out a group of what might be considered 'great works' of fiction. (And which might include SF as well, such as by Atwood, Orwell, Bradbury, Huxley, and so on; not to mention high fantasy like the Bible or the Odyssey). An unstated corollary is that in some sense it's all -- or only -- the smaller focus, contemporary realist stories that have the right to the label 'literature', but if so that's unwarranted.
Genre is a way to market written works so that people can find what they like. There's no mystery to this. People's likes tend to fall in categories. And though no single system will perfectly fit them all, some system is better than none. So long as we don't take it too seriously.