assuming you haven't made everything func_detail (please don't
), how much func_detail have you got? As mentioned by bubblewrap, but there's a bit more to it. I don't mean to confuse you, so if the following does, just ignore it.
Basically, a func_detail is treated by the compiler as a prop (I believe it compiles them separately and places them into the bsp after creating the world). You need to use them when you have a brush that you don't want to cut up vis (both, visibility and the compile process
). These cases include stairs, stuff that you can see through, and most other details or things with large amounts of geometry relative to their size.