If that counts for every map, why do they even bother about those minimal diffirences? As when teams have to switch every unbalanced part gets countered.
It as far as i know only matters for linear maps that dont require the teams to switch sides (linear 5cp, ctf).
But even then i still think its important to know on which parts they look to decide if its balanced.
They switch sides on every map (at least in ESEA and UGC), both for the purposes of taking a break (restroom run, phone calls, etc) and also for balancing (as in koth_viaduct). Its mostly a holdover from CS, where the attack/defend nature of ts vs cts gets built in to ESEA's matchmaking.
The difference between koth_viaduct asymmetry and cp_granary assymetry is quite a bit more extreme than you are making it seem. In an original version of granary, the red warehouse for the second point had some very slight metal trim on the wood pillars. Scouts could use that trim to jump from the floor up to the catwalk areas, and get into a position in like two seconds as opposed to like ten.
The viaduct changes, on the other hand, are largely down to rocket paths, as rockets fire from the right side of the screen and can get cut by level geo on the different sides of the map. While that does affect gameplay, its usually a matter of 5 - 10 units you can walk to negate the sightline (a change of about 1 second, or even less depending on the angle). Rocket jumping is also affected, but its the same basic idea: you might have to change your position a bit to get the same sort of jump, but its almost never impossible to get the same basic trajectory.
Ultimately, while you might complain that competitive players are inconsistent in their demands for symmetry, non-rng, balance, etc etc, you have to understand that you are talking about a community of players, many with their own wants and demands, some of which will end up contradicting. But, the demand for symmetry in level geometry is ultimately far more consistent among competitive players, so if you are planning on making a koth or cp map that you would like comp players to play, you increase your chances tremendously by avoiding asymmetrical geometry in places where it matters. If you think thats BS or inconsistent, then you don't have to do it, but its kind of ridiculous to expect competitive players to budge on this issue, especially when the logic of their demand is so well developed (assymetric layout gives players an advantage in an unfair way, level designers can adjust level geometry to be consistent, competitive players should demand symetric geometry).