Reasons to Keep Demi-Humans in your Game
Because they're fucking classic! How can you have a fantasy game without the eagle-eyed archer-elves prancing around their treetops or dour-faced dwarves laboring in their forges beneath the earth? Players need to understand your fantasy world in order to interact with it logically, and they (probably) already know the basic tropes. Elves and dwarves are so ubiquitous that everyone already knows not to get in a drinking contest with a dwarf, or challenge an elf to a contest of who-can-live-longer. Familiar tropes allow players to dive into the action immediately.
Of course, I think that bullshit runs rampart in the last paragraph, for reasons that I will enumerate in the next.
Reasons to Get Rid of Demi-Humans
You've already seen dwarves and elves a million times. They are tired and they are cliche. This can be a strength (see last section) but I think it is a fantastic weakness.
Fantasy games thrive on "fantasy". The unknown. The extraordinary. The weird and the fantastic. Every time your players run into a squadron of dwarves out reclaim their (another) ancestor's lost fortress, you are missing an opportunity to replace the dwarves with something weirder and more fantastic. Don't get me wrong, helping dwarves reclaim their lost fortress is fucking awesome ("They dug too deep. . . they dug too deep!"), but why not help a bunch of mosquito men find their lost wife? She's succumbed to egg madness, and could use rescuing.
Mosquito people are people, too, dammit.
I blame Tolkien. His influence can be felt in how we still use his spelling ("elves and dwarves") instead of the previous dictionary version ("elfs and dwarfs").
Elves: graceful, intelligent, sensitive, archers, wizards, long-lived (or immortal), etc. They live in Art Noveau treetop houses and sing songs to the trees. Sometimes stuffy, xenophobic, soft-spoken, frequently English accent, etc.
Dwarves: short, stout, Scottish accent, BEARDS, underground, warriors, vast halls, miners, metal, gems, booze, and they hate those sissy elves. Axes, boasting, unimaginative, industrious, etc.
But I'm sure your dwarves/elves are different!
I'm not surprised. Given enough exposure, the human brain recoils from cliche like an octopus dropped on a hot frying pan. Maybe your elves are mysterious, or they're plants, or they're dying, or they steal babies, or they live in the Feywild, or they're whimsical creatures of nature. Maybe your dwarves are made from stone, or maybe the women have beards, or there are no dwarven women, or they're tinkers, or bankers, or runecasters, or steampunk golem-caressers. This is good, but you're still just repairing the problem, not eliminating it. And the weirder you make your dwarves and elves the more you sacrifice the comfort of old tropes. (Dwarves are like sweatpants.)
This sort of compromise is probably best for the average campaign. Keep enough to be familiar but bring enough just enough to keep it fresh. Some people like playing a dwarf so they can be surly and speak in an easy accent. Some people like being elves so they can be better than everyone else.
But why not get rid of them all together? Or have them as distant, mysterious races surrounded by myth and rumor? Put the unknown back into dwaunknownrf!
Psychology: We Are Controlled by Brains!
Elves, dwarves, and orcs persist because they are nerdo-Jungian archetypes. They represent a specific prototype in our collective unconscious. There is only one dwarf, and all other dwarves are merely reflections of this Platonic dwarf, this Ur-dwarf, this ideal collection of symbols that code for dwarfness. The Platonic dwarf has survived this long for a reason. Whatever fantasy race you create will not persist in the universal human psyche as indelibly as these guys.
This is also why you rarely see female orcs or dwarves. The ideals they represent are (traditionally) masculine ones.
To a lesser degree, this is also true with a lot of monsters. Morlocks and vampires come to mind. The point is that these are thinking, reasoning, feeling, talking creatures that are just as intelligent as humans. If elves civilization is way older than human civilization, they should be way more diverse. And yet all elves are shoehorned into same monoculture in most settings. And yet humans have duelists, dervishes, paladins, and berserkers. This is the same human-centric conceit that ensures that humans will always be the "middlest" creatures in any setting--compared to all the other races, humans will never be the "strong race", the "weak race", the "smart race", or the "dumb race".
My Elves are Different
Only a jerk talks about problems without proposing a solution. And it's boring to read a long post without anything interesting. So here's one idea for the different races that I have (based on the idea that we are dealing with primitive human archetypes).
Dwarves were created to be drudge laborers. Especially mining asteroids. They live a long time because less time is lost to training. They don't mind the temperature, they don't mind the dark, and they get along fine with low oxygen content in the atmosphere (although it does make them talk funny). They don't mind high-gravity because they have black, crystalline bones that are heavier than lead and stronger than steel. They have a pathological NEED to work, or else they go crazy. Their constructions are either unadorned Brutalist architecture or covered in some simple, ultra-intricate pattern that only serves to show that millions of man-hours went into the construction of this facade. Like the one really stoned buy in art class that just covered a huge sheet of paper in tiny, perfect squares.
Half of all dwarves are female, and they form stable, family-oriented homes. They work just fine in the open air, but their unique racial abilities make them better at mining than anyone else, so they tend to do that. They work just fine in the low lands, but their biology makes them well-suited for mountainous heights, so they tend to live there. They hate idle time. They produce no non-functional art and they are not curious because laborers shouldn't produce art or be curious. They drink beer and sing songs because that's what you do after a long day of work. Their songs are simple things, not much more complicated Work songs help them work better in unison. These things are in their blood. All dwarves are the same in these regards.
Orcs were created to be soldiers. They don't live a long time because they have a +50% accelerated metabolism; when you want supersoldiers you want them NOW. A 12-year-old orc is as mature as a 18-year old human. A 40-year-old orc is as wizened as a 60-year-old human. They have a pathological NEED to fight, or else they go crazy. They are really strong and can run for days without tiring. They aren't smart or creative because they are supersoldiers, not supercommanders. One one level, they actually enjoy organizing and taking orders. They can be surprisingly loyal if indoctrinated early and commanded by a legitimate authority figure (not just some orc who became warlord by killing the last warlord).
Only about 5% of orcs are female, but they are super-fecund and birth litters of soldier-pups. Only the alpha orcs will have orc-wives. Orcs actually have decreased sex drives compared to humans, but some of them still engage in wife-stealers and/or homosexuality. Either way, it's transgressive and all orcs must be killed, because they're obviously not people.
Halflings were created to be spacers. They were made as small as possible, because smaller spaceship pilots use less resources but work just as well. They have terrific difficulty with pregnancy and childbirth because tiny hips and it's tough to make a baby any smaller than it already is. (Human fetal brain development is amazing stuff.) Cesarean sections were intended for halflings, and vaginal births (the norm) are horrible, painful, and risky for everyone involved. Since breeding is hard, they have a hard time rebounding from population declines. But since farmland can feed eight halflings for every human, population density is high in halfling lands. This is especially noticeable in the cities.
Halflings don't have any deep-seated genetic imperative. They're just miniaturized humans, with all the foibles that entails. Because it is easy for humans to tease, rob, or kill a halfling, the ones that live near humans have an exaggerated form of masculine bravado where they are easily insulted and quick to anger (and violence). If you wrong a halfling, they'll show up with their entire clan (15-30 halflings) ready to kick your ass. They favor spears (better to fight the big people with) and crossbows.
Elves were created to be superior rich people. When the rich people wanted to ensure that their offspring would be the best possible creatures, they engineered them with intelligence, charm, grace, dexterity, and strength. Aside from being better at everything, elves are universally beautiful and have all sorts of vanity genes built into them, such as violet eyes, iridescent skin, musical voices, and rainbow hair. They are all clever and creative. They are tall, graceful, and never get fat. They never suffer from acne. Incredibly, their sweat doesn't smell bad (they have no skin pores for bacteria to live in) and their shit literally does not stink (their intestines are hostile to bacteria). Elves meeting a human for the first time have a hard time believing how smelly they are. Like their predecessors, they are predisposed to smugness, sociopathy, and a powerful sense of entitlement. They look eternally youthful and are incredibly long-lived (they were originally designed to be immortal, but the techniques for this have been lost).
Their only weakness is poor health. The pursuit of vanity made creatures that are forever dependent on supplements and controlled environments. Not having body hair or sweat glands (they have an inferior cellular method depositing some slight moisture on the skin) means that they overheat easily. Not having intestinal biota means that they have severely restricted diets (i.e. meat makes them violently ill), and can only eat elf food (they are like severely auxotrophic humans). Their beautiful skin is slow to heal, their inability to form callouses means that it blisters easily. Their difficulty metabolizing fats means that their joints are easily injured. Their cellular processes that prevent aging make them prone to childhood cancers. They have a myriad of health problems, which they hide from outsiders and mitigate with daily cocktails of exotic drugs and nutrients. And still they sicken and die easily when they leave their super-clean enclaves. Also they are hated by 99 percenters the world over.
Because elves are rich. Like how the children of rich people are likely to be rich themselves. Compounded by racial favoritism. Like doctors and lawyers, most elves can make their money as individuals exploiting information asymmetries. Famously, elves are wizards who make and sell magic items for small fortunes, but they also work as doctors and sages. Also famous but less wealthy are the elven artists and bards, who will also never know poverty.
Why do elves live in forests? Elves like to live in beautiful places. This isn't a genetic thing. It's social constructionism. Elves see themselves as superior and beautiful, therefor their cities must be superior and beautiful. In a group of people, the most attractive one is more likely to put more effort into their appearance. So elves will try to live anywhere that is (1) away from the smelly humans, and (2) beautiful, because they themselves are beautiful. Treetops, cliffs, mountainsides, etc.
Please note that dwarves shouldn't be presented as laborers and elves presented as the elite. Just present them as dwarves and elves and rework them however you want.
These are just outlines of biology. Every race listed above has many different cultures and languages. Where there isn't egregious racism there will be galling racial favoritism.
Assigning roles based on genetics seems odd, because then do you have elves commanding armies of orcs? Do dwarves happily sweep the streets of halfling cities? Not in the current pseudo-medieval climate! Xenophobia, racism, and pride dominate interracial politics. Everyone would rather divide along racial lines and kill each other.
Also, elves are such goddam Mary Sues.
Also I now feel like a giant racist for writing this, and these aren't even real people. My liberal heart bleeds Catholic guilt.