[PSA] Don't Test With Bots

Discussion in 'Tutorials & Resources' started by tyler, Jun 27, 2012.

  1. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,035
    Positive Ratings:
    3,990
    DO NOT TEST WITH BOTS

    I see a lot of people who are new to mapping say they tested with bots. It made me wonder how many people are testing their map and making changes based on bot behavior.

    This is a really, really bad idea.

    Bots are stupid. Even on their highest difficulty, they are stupid. They come nowhere near a human player in terms of AI. They choose classes randomly, without thinking about team composition or previous class--so an Engineer with a full base set up might randomly change to Heavy. They build and snipe in arbitrary spots--unless you bother to edit the nav mesh that is generated for your map.

    They don't even jump.

    That includes rocket and sticky jumping. That means no double jumping Scouts, either. And you know what else? They won't walk off anything higher than about a hundred units. Bots are stupid. (Their code is based on the AI for zombies from Left 4 Dead. Does that explain their stupidity yet?) This also means they always take the stairs--they will never just hop up a short ledge to grab health or ammo.

    All this means that if your change your map based on what the bots do, you're bound to be heading in the wrong direction. You don't want your map in rotation on servers where only bots play, right? Then test with humans.

    They aren't your only option, either.

    TF2Maps (this website! the one you're on!) runs tests of custom maps all the time. It is free to join them and get your map played by real, human players. No bots, ever. Just watch the Events forum for game day announcements or other events and submit your map. You don't even have to be there! STV demos are automaticaly recorded and you can watch them later.

    You can also join the Steam chat room and hope to be included in an impromptu test.

    Bots are good for one thing, though.

    It's fine to use them if you want to check your map's logic to ensure all doors and timers are functioning properly. But you should be able to do this yourself, alone. Just remember, whatever you do, don't pack the nav mesh into your bsp!
     
    • Thanks Thanks x 16
    • Agree Agree x 1
    Last edited: Aug 30, 2016
  2. Trotim

    aa Trotim

    Messages:
    1,180
    Positive Ratings:
    997
    sometimes I load tf2 only to play with bots on koth_trotim1

    foreveralone
     
    • Thanks Thanks x 13
  3. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    At least there are a few upsides to bots:
    • They don't forget to play the game. Usually.
    • They manage to find routes that humans don't instinctively find.

    But this hardly compensates for all the bad points you already brought up.
     
  4. Spacek531

    Spacek531 L4: Comfortable Member

    Messages:
    162
    Positive Ratings:
    60
    I am curious why we are advised to not pack the .nav file in the .bz2. Is it for a space reason, or is it because with a .nav, server ops may spawn bots and mess up everyone's experience?
     
  5. Wilson

    aa Wilson Burial by Sleep

    Messages:
    1,256
    Positive Ratings:
    841
    If bots are better at finding their way around your map than players, then there is a problem.
     
    • Thanks Thanks x 3
  6. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,131
    if you pack a .nav into the map, you mess up everyones' experience. including the server op.
     
  7. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,906
    It crashes servers and clients.
     
  8. fubarFX

    aa fubarFX The "raw" in "nodraw"

    Messages:
    1,629
    Positive Ratings:
    1,715
    you can't override a paked nav file. if a server operator or w/e would make a better nav for your map, they won't be able to use it on their server. also, if the AI is ever updated and need new navs, you will need to release a new version of your map. keeping your nav separated is just more convenient.
     
  9. Tarry H Sruman

    Tarry H Sruman Large Orphanage Proprietor

    Messages:
    871
    Positive Ratings:
    820
    We should petition Valve to have this message appear onscreen every time bots are added to a non-stock map.
     
  10. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    Yes, because players are dumber than bots when it comes to finding routes. At least while accounting for that bots are limited at movement.
     
  11. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,906
    I respectfully disagree, in one sense. Bots can "see" all the routes, and choose which one they think is right. But this is not what you want.

    Humans choose what they think is right, but can't see all the routes. Humans are the ones you want to testing, because those are the ones playing. There is no way that a TF2 Bot is better than a human player, with the except of following commands.
     
  12. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,745
    Ehhh Bots have uses. This is one of those things where if you're the 'target audience' for this message, it applies 100% and if you're not it applies about 0%.
     
  13. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    You are exactly right, it's the players that will play. And if they don't see the route then you have a problem.
     
  14. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,906
    Not exactly what I meant, and sorry, I was unclear.

    Players can see routes, but they can't see/know about it Instantly, like a bot can. Learning is the big thing you need in getting good feedback. You can't get a good supply of feedback, if the tester knows EVERYTHING. Yes, you'll get feedback, but it won't be as good as a real player.


    A Real player is ALWAYS better than a bot.
     
  15. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    I think the point here is that a map should be easy enough to learn. In my experiences it is closely related to being easily navigable, but not the same. You could make a map navigable by throwing signs everywhere.

    Players are humans. Humans aren't rational. Humans aren't perfect. They will miss obvious stuff. They will notice trivial stuff. They fail the easy and makes the difficult look easy.

    Players need a different kind of gudience than bots. Bots can instantly learn all the routes. But only a player can learn how the routes work. But at the same time players need to be able to play the map while learning. Some would say that a player never stops learning. Players learn what other players do and change their behavior in response. It's a never ending battle of wits (and some guns too). Bots do nothing even remotely close to that.
     
    • Thanks Thanks x 1
  16. Prestige

    aa Prestige im not gay anymore

    Messages:
    1,774
    Positive Ratings:
    1,530
    the main thing is sentries. bots cant take down sentries at all so any engy that isnt going against you will screw shit up.
     
    • Thanks Thanks x 1
  17. Trotim

    aa Trotim

    Messages:
    1,180
    Positive Ratings:
    997
    limit engineer to 0
     
  18. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    The issue wasn't that they couldn't play as engineers (which they only suck at), but that they are bad at attacking engineers.
     
  19. xzzy

    aa xzzy

    Messages:
    815
    Positive Ratings:
    393
    This may not be 100% true anymore. A map I'm working on has a one way route with two drops, each about 200 units. I've seen bots willingly walk off these ledges on several occasions recently. The only caveat is that in my case the drop is the only option.. it's either jump or head back to spawn, and their code probably prefers moving forward.

    Not that this makes bots suddenly awesome map testers, they aren't, but I think someone at Valve still works on the code and occasionally implements tweaks.
     
  20. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,035
    Positive Ratings:
    3,990
    I think it mostly has to do with how their meshes are generated. In that case it'd tell them to just keep going, but in most cases it'd tell them to prefer the stairs or the ramp or whatever.