Best way to put a huge section of a map underground?

I suggest using floors linked by alternating ramps if you're really set on 3500 units. In my opinion, though, 3500 seems way too much unless you plan the underground area to be huge horizontally. If you have no specific reason for such a massive distance, you may want to consider 512 or 1024.

3500u height difference wont work, you will have incredibly long stairs/ramps, and please dont try to use elevators D:

Also, you could just place the cave area a little bit lower than the rest of the map, but put a mountain or something above it.

Elevators shouldn't be used for the same reason ladders shouldn't be used. They restrict movement and aren't used in any Valve maps.

Why not just make the mountain or whatever it's in taller?

I have had an experience with large height differences myself, and ran into similar problems. While my wall was little over 800 units, I still noticed how incredibly tedious it was to traverse. Being outdoor, my solution was that players could either take the tedious stairs, or simply jump off of the top into a puddle of water. The other trick was to put second resupply areas into the bottom of my structure, so that while players still had to get there, they could resupply easily once there. So, while the whole walking through a waterfall idea is good, if you want to get a bit gimmicky you could have them jump off of some scaffolding, landing into a high-pressure air passage, which would catch them and give them a soft landing.

Make "mine teleporters" maybe? Yknow there's this function when you enter an area you get teleported somewhere else that's used in some idle maps?
Make a sort of mine shaft that goes straight down, with a little platform and a nice teleporter particle effect on it, and when you step on it you get ported right down.

OR, make it look much longer than is actually is: Really long tunnel, with a teleporter effect a little bit down from the top.

Or you could have the red spawn for the first stage be at a mining elevator, with the blue spawn at the beginning of the second stage at the bottom. Implied between-stages transition! It could work.