I'm with you, to an extent. I love the Narnia books, and I love CS Lewis's forthrightness in saying without fear what he actually thinks. However, I think you may be doing the inverse of what you criticise others for. Are you really saying that Lewis intends us to come away with a positive view of Calormenes? Yes, there are individual good Calormenes, and yes there are praiseworthy aspects of their culture. But overall, the plain sense you get from reading the books is that Narnians are the goodies and Calormenes are the baddies.
Lewis also has some rather archaic views about boys and girls which seem out of place today. Some of them I think he would repudiate if he were still alive today. But given that he was in his fifties when he wrote these books some sixty years ago, and was fairly old fashioned in his views even for his time, I think we can forgive the fact that his views seem a bit nineteenth century! It doesn't spoil the books for me - actually, it enhances them. I don't think he is uncharitable. I don't think it is fair to brand him a sexist or a racist, given the time he was writing and the culture he grew up in. But if he expressed the same views today I think he probably would be sexist and racist, or at least bordering on it.
Peeps
I think that Lewis intends for us to come across with as balanced a view as possible of the Calormenes given that the Calormenes are set up as antagonists to Narnia in
Horse and His Boy and
Last Battle . Frankly, based on what is in the text, we could almost as easily make the case that Lewis didn't like white people, because the White Witch is very pale and she is basically evil incarnate. The Telmarines are also white and in PC, the Telmarines are depicted as the antagonists with only a few individually good Telmarines (Caspian, Nurse, Gwendolen, etc.) I don't find that much difference between the treatment of the white Telmarines in PC and the dark-skinned Calormenes in
Horse and His Boy and the
Last Battle.
I also think that we are more meant to feel dislike for the invasive policies of the Calormen Empire and for the men who order those invasions (i.e. Rabadash), but I never felt like we were supposed to hate all Calormenes or even all the Calormene soldiers.
Yes, they were the enemy empire, but many fantasy stories have an enemy empire, and most fantasy stories never get around to showing good members of that enemy empire, and most of the time people of different countries belong to different races. There's nothing really surprising about that, in my opinion.
Also, for me, even if the Narnians are the good guys and the Calormenes the bad ones, I don't see that as being racist. Most of the Narnians, particularly in
Horse and His Boy, are mythical creatures and talking animals. We could, again, almost as easily get the sense that Lewis is anti-human and he's only focusing on a couple of good humans (whom he compliments and critiques) to distract us from that negative message.
As far as the sexism charge goes (which is probably off topic, so I shouldn't have brought it up, lol) I guess I see most of the remarks that people label as sexist as either being part of funny little Battle of the Sexes arguments that people don't really take seriously, as jokes, as a way of characterizing pubescent children who are normally quite advanced for the cooty-stage since most of them have friends of the opposite sex, or as being voiced by characters we aren't meant to like such as Uncle Andrew. Most people who accuse Lewis of sexism just seem not to have gotten a punchline and stomped off in a huff. Chivalry is lauded, but what can we expect from a medieval world?
Honestly, I would rather read about Lucy, Jill, Aravis, and Polly as written by "sexist" Lewis than read about typical "feminist" Mary Sue fantasy girl who is beautiful, funny, smart, fiery, strong, and all-around perfectly unrealistic who is all too common in the genre today. I'll also take his fairly balanced guy-vs-gal cast over the token female common in stories and movies of today. By and large, I actually see a lot more sexism in today's literature (where the woman must either be perfect at both male and female skills to be worthwhile, or else she must take on traditional male roles in order to be interesting).
I do think that Lewis might have written his books differently if he lived in today's society, and that, living in a different world (or rather time in our world), he might have written about relationships between sexes and relationships between races in a manner that reflects modern thoughts. I don't know if he would have agreed with modern thought on everything, but I'm sure, at least, it would have influenced how he chose to write. However, by and large, I don't think that he wanted anyone to come away with the idea that white people are the master race and girls are silly wastes of space. I think within the constraints of his stories and time, he was, on a whole, trying to portray different races and sexes if not in a positive way at least in a balanced way.
Anyway, that's just my two cents, and I don't mean to offend anyone. I just think my opinions are more right than everyone else's

