I'm gonna answer the questions in reverse order because I'm cool like that.
The player count for medieval maps shouldn't be so bad that it's impossible to test the map. My own medieval map got its fair share of testing when I was working on it. That multistage CP medieval map that someone was working on recently got at least a few fully manned tests on our servers. A lot of people who join our tests compromise personal tastes in maps for the benefit of testing, be it crits, a certain gamemode, or just a bad map, we suffer through them. I don't see why medieval would be any different.
There are exactly two server configurations that need to be considered when testing medieval maps: the text chat filter (tf_medieval_autorp 0/1) and random crits (tf_weapon_criticals 0/1). The medieval chat filter is a little amusing, sometimes, if it's your first time playing medieval mode, but after the novelty inevitably wears off you find yourself wanting to actually understand what the chat is saying. You can bypass the filter by putting an exclamation mark at the start of your message, meaning plugin feedback is unaffected, but this is a little-known and awkward way to go about it - I recommend we turn it off. As for random crits, medieval mode's heavy reliance on melee weapons means crits happen a hell of a lot more often than normal TF2, so they have much more opportunity to affect the outcome of tests. Like we do with Arena maps, I recommend we turn crits off for any medieval map.
As for judging a medieval map, it's different from normal TF2, I'll give you that. While what others have been saying apply somewhat, in that medieval has its own set of judgement criteria the same way every gamemode does, the problem is that medieval's judging criteria is not clearly defined. Medieval maps are few in number, played even less, and traditional wisdom doesn't carry over as well it does with other gamemodes. BUT. I feel that disallowing it from contests is only exacerbating that problem. Medieval is shit, and is always going to be shit until we figure it out the way we have done for other gamemodes. The stigma against it in this community and the TF2 community at large is a big enough hurdle to get over without it being disallowed in every contest.
Speaking from personal experience, I've had a couple ideas for medieval maps kicking around, but contests are what initially motivate me to start mapping. The fact that any medieval map I want to enter into a contest is relegated to the 72hr, where the elusive quality needed to revitalise the gamemode is generally quite limited just defeats the point.
If judging criteria for medieval maps is a concern, then we should be encouraging people to make more medieval maps, so that we can learn more about the gamemode. Disallowing medieval maps from contests is directly counter-productive in that sense.