Depends on how much importance they have on the overall gameplay of the map. One thing I've found is, the more you deviate away from "normal" TF2 core objectives, the less of a positive reception you'll get.
Phys objects in TF2 aren't the most reliable things, and 99% of the time you'll need to force alterations to cvars to get them to work the way people would expect them to, and even then some weapons/melee cause them to behave strangely. Now, that's not to say you shouldn't make a phys object the crux of your map- I've spent the last decade plus making Pipeball maps- but some people will just absolutley hate it for only the inherent jank that comes with physics objects in Source
As for breakables, if they're just limited to cosmetic objects that have no bearing on how the game is played, people will chuckle and move on. If it's central to a map's objectives though, most people have a dim view of "Generator" maps.
If there's something you want to do, just go ahead and do it. Sometimes it'll take off, sometimes it won't.