"Speculative fiction" is hard to define, mostly because nobody agrees on the meaning. Broadly, there are two useful definitions, but to understand them, we have to take a brief (BRIEF!) look at the history of science fiction.
1. About 100 years ago, people called science fiction a thing.
2. About 80 years ago, sci-fi hit what's considered it's "Golden Age."
3. About 60 years ago, a LOT of people were writing sci-fi. Not all of it was good.
(Told you it was brief.)
It was around this time that Robert Heinlein coined the term speculative fiction, and gave it its first definition:
speculative fiction: (n) 1. Fiction that has science-fictional elements, but is not science fiction.
Here's what happened. When sci-fi got big, it also got stereotyped. It became seen as cheap entertainment for the masses. "Genre" fiction as opposed to "real" fiction. Critics treated it as subpar literature, even though (and I love this quote from Peter Watts) "The same critics who roll their eyes at aliens and warp drive don't seem to have any problems with a woman ascending into heaven while hanging laundry in One Hundred Years of Solitude, just so long as Gabriel Garcia Marquez doesn't get published by Tor or Del Ray."
Ever since then, a lot of sci-fi authors -- like Robert Heinlein, Kurt Vonnegut, and Margaret Atwood -- have tried to distance themselves from the sci-fi label. They often use the term speculative fiction to do this.
But it's not a very useful definition. For one thing, it defines itself by what it is NOT, which is silly.
But also, it's arrogant. It tries to define speculative fiction as "science fiction, but good." It's an offense to Herbert, LeGuin, Asimov, Card, and thousands of other genuinely good sci-fi authors who weren't afraid of the term.
I think people realized this, but the term has stayed in use. But to most people, it now mostly means this:
speculative fiction: (n) 2. An umbrella term covering everything from science fiction to fantasy to magical realism.
At first glance, it appears too broad to be useful. Almost like saying spec-fic is any fiction that could not have occurred in the world as we know it.
Two things make this definition useful: (1) fans of sci-fi and fantasy* have a large amount of overlap. (2) A lot of speculative fiction does not fall easily into one of these subcategories.
Speculative fiction gives us a way to talk about works like Miéville's Perdido Street Station without having to decide whether the fantastical races make it fantasy or the high-tech, steampunk elements make it sci-fi. Or whether Gene Wolfe's Book of the New Sun is fantasy because it feels medieval or sci-fi because it's dystopian and post-apocalyptic.
It is almost too broad a term (which is why I didn't use it in my query), but it's inclusive rather than snobbish, which I much prefer. Instead of saying, "That can't be genre fiction because it's not garbage!" I'd rather say, "Yes, this is genre. Some it is actually GOOD."
* And magical realism and horror, paranormal, dystopian, post-apocalyptic fiction,
superheros, alternate history, and everything else spec-fic usually covers.