If you're behind a router you're safe. Unless, of course, you've intentionally forwarded the TF2 server ports to your computer.
I had the same happen to me.
Initially this was not happening.. Then I set up a dedicated server on my machine and forwarded the port in the router. Next thing I know I am playing around tossing prop_physics_multiplayer stuff with stickybombs and a sniper shows up and pops me in the head... then proceeds to ignore everything I am saying trying to explain what is going on while he keeps popping out comments like 'WTF IS THIS?' 'WUT IS THE POINNT OF THIS' 'WUT R WE DOING?' (caps his). All the while I am explaining it was just a simple test to play with tossing around physics objects with stickybombs. Unfortunately I had to kick him, as he appeared to be too dim to cooperate with me on anything and just kept shooting me.
I always set a password now lol.
And I have no clue what you all are talking about Got a basic wireless connection, Avast, windows firewall (I know, but can't afford a real one), and that's it. Never once had people join my test maps, though it would be kinda interesting.