I think it could help if the passageways to each point from E reflected the look of the point they go to.
That's kind of what I'm talking about when I say the map is confusing. It's not a total labyrinth, but I never get the sense that I'm moving from one area into another. In fact, I'll just come right out and say it: when I'm playing, I feel like the layout is there to serve the art design rather than the other way around. I don't think that's actually how you made the map, but that's how it feels when I'm going from point to point.
The main issue, for me, is that the art design is so well conceived as a conceptual whole, i.e. a gothic castle with a big Frankenstein electro-lab in the middle, that the transitions from one point to the next don't really progress in terms of art style as much as they simply shift from one setting to another (say, graveyard to chapel).
To put it another way, if you go into a real castle, that's exactly how it feels: you go through one stone hall after another and suddenly you emerge in a dining hall or a library or what have you. In most TF2 maps, though, the textures, colors, shapes, etc. all gradually change to indicate progression, and this necessarily comes into conflict with the whole idea of a castle.
Just to reiterate, I have basically the exact same problem with Steel, but the art design in your map makes even the large environments claustrophobic and I always feel like I'm just wandering aimlessly even when I know exactly where I'm going.
TLDR: I honestly really love the art design and how well you made it feel like a castle, and the map does actually play well, but I always feel lost and trapped because, ironically, it's like actually wandering around inside an old castle.