It’s all about control, and how to achieve it.
I think we all understand pretty well how science and technology work. Even when we postulate something like FTL, which is contrary to our present understanding of the universe, we get to it rationally. We don’t get there by lighting candles around a pentagram.
Magic is another world, an alternate way of achieving control, and I see it falling into three types, with quite a lot of overlap.
First, there are the unconsidered, slobbering monsters of gothic horror and B movies. They just are. If they are the offshoot of a mad scientist or a nuclear explosion, we call them science fiction. If they are the offshoot of an ancient curse, we call them fantasy. Frankenstein and Dracula are examples of alternate strands, but really, they belong together. Their effectiveness is on our fears, not on our need to explain. And, no matter how the monster is overcome in the end — if he is — the experience is all about not having control.
Second, there are magical systems which are simply science under different laws. Do this and that happens, just because that’s the way things work in one projected universe. Nobody did this better than Randall Garrett in his Darcy stories.
Third, there are supplicatory systems which assume that there are forces in the universe, and some of them are personalities. It is very transactional. Do something for the power, and the power will do something for you. But beware of the fine print in the contract. Every variation of the Faust legend falls into this pattern.
Many stories take on the colors of more than one system. In both Harry Potter and A Wizard of Earthsea there are spell that just work, like E=MC2 works, but there are also callings which evoke powers who are personalities, and usually not very nice ones.
These are logical systems, but nobody cares about them until we embed them in stories.
# # #
On the question of science and technology vs. magic, a scholarly type could easily expend a thousand pages. That might be a little dry, though.
In our everyday lives, we all use science to make our lives easy, but there are some questions that science can’t answer. Which questions? We decide that for ourselves, and the tipping point is different for each person. After all, you can use your new self-driving car to go to the psychic and have your future told, if you are so inclined.
The same balance is found in stories of fantasy and science fiction. It is rare for a science fiction novel to completely lack the unexplained and unexplainable. Luke Skywalker had the Force. Every mutant in the 1950s had psi powers. In Jandrax, I put in a possibly Godly encounter because the book needed it. Most science fiction novels have a touch of magic somewhere.
On the other hand, novels full of magic still have physical laws. The sky is up and the ground is down. If you fall off a high cliff, you die. Of course you might use a levitation spell to save yourself — just like one of those mutants in 50s science fiction would levitate by their psi powers — but you wouldn’t need to if nature wasn’t trying to kill you. The physical reality of heat, cold, hunger, thirst and a fragile body are as much a part of fantasy as they are of science fiction — or romance novels, spy stories, or detective tales. It’s a long way from the Shire to Mount Doom if you have to walk. Frodo could have used a Lear jet.
The places we go in science fiction are great fun and sometimes scary.
In fantasy, we go places that are sometimes great fun, but usually scary.
Which journeys we take are a matter of taste. I would love to sail with Ged by a mage wind, or by the world’s winds, but I wouldn’t read H. P. Lovecraft on a Romnean ten thousand dollar bet.