I had similar issues on my MK maps and found that when I had resized some objects I sent them completely off grid making all sorts of very small portals that needed to be cut by vvis. I spent half a day just going through and lining up everything on grid on all 3 Axis and it went from an 18 hour compile time down to 120 mins or so and then with further tweaking and replacing some bad brushes and gracious usage of func_detail I am down to about 1 hr and 23 mins.
One thing I found very handy when working with func_detail was to change the view options to show everything EXCEPT func_detail brushes. From there I could view the map via the 3D view and figure out what more I could convert to a func_detail. IE Windows, Poles, small walls that players can hop over, railings, trim etc etc. Anything that is a not square and can be walked or seen around I typically also func_detail and this has made a big difference with roof overhangs or pillers (mushrooms and cannons in my case)
And if you have not already load the maps portal file while in the 3D view. It will give you a very good look into where you need to clean up brushes or find more ways to convert the brushes to func_detail or func_brush or similar.
-Teddy