You'll get a "Server map differs from client" error. - I asked Jill (who's responsible for the packing implementation in TF2) if it'd be possible to fix and he said it wasn't in their immediate list of priorities because it's a very low-level change that might have unforeseen consequences.