TF2 Map Workshop Server Configuration

Discussion in 'Team Fortress 2 Talk' started by worMatty, Jun 4, 2015.

  1. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Information About Hosting Workshop Maps on a TF2 Server

    Hi, folks.

    This thread is for discussion about server configuration for Workshop maps. It does not cover publication or editing for map authors, or client problems. For that, please see this thread. This post is updated when anything significant changes about hosting Workshop maps on a server, after it has been tested and confirmed as best it can be.


    A Bit About Workshop Maps

    Workshop maps can be hosted easily on both stock TF2 servers and on servers that use SourceMod. You do not have to download a Workshop map manually to upload it to your server. The whole system of obtaining maps, and updating them, is handled automatically by the server in conjunction with the Steam Workshop. Clients connecting to your server will download the maps from the Workshop, making your FastDL server redundant.

    Each Workshop map has a unique nine-digit number, which I call a UGC ID. You can find this in the URL of each map's page in the Steam Workshop. You must use this number to load maps, and in your mapcycle file.

    Each Workshop map that has been downloaded to your game server is stored in its own directory in steamapps/workshop/content/440/{ugcid}. You can specify the directory where Workshop maps are stored using the launch parameter -ugcpath. The file steamapps/workshop/appworkshop_440.acf is a list of all the maps you have downloaded. The server uses it to keep track of them. You can use the cvar tf_workshop_map_status to check the currently tracked maps and their status.

    Workshop maps are already compressed (known as repacked), and are slightly smaller than if they were compressed using BZ2, so if you make the switch to them, you can reduce the amount of disk space used.


    How to Run Workshop Maps on Your Server

    To run a workshop map on your server, just add it to your mapcycle using either of these two formats:
    1. workshop/123456789
    2. workshop/cp_mapname.ugc123456789
    :hint:
    The second format includes what I call a 'friendly name' for the map. This does not have to be exactly the same as the map's filename (sans .bsp extension). Rather, it can be used to help you when you edit your mapcycle. The friendly name is also what SourceMod displays in its menu system, so try to keep it brief and informative.​

    When your server starts, it will read your mapcycle file and resolve all of the UGC IDs. It will begin to track them, so that if an updated map is available, it will be downloaded. To see the maps your server is tracking, you can use the command tf_workshop_map_status. You can tell the server to check tracked maps for updates using tf_workshop_refresh.

    There is a short delay for connecting clients whilst the server downloads a map for the first time, during which time they will not be told what is happening.

    If there is a dead map in the mapcycle (that is, a map that is no longer present on the Workshop), then it will still appear in voting lists, and can be voted for. If the map wins a player vote, then when the server attempts to switch to it, it will instead revert back to the map that was last played. The server console will output an error about such maps on each map change:
    Code:
    [TF Workshop] Error fetching updated information for map id 454372402
    [TF Workshop] Info lookup failed for workshop file 454372402 ( EResult 1 )
    If the server attempts to change to a dead map as part of its cycle, it will fail and try the next map in the cycle:
    Code:
    CHANGE LEVEL: workshop/454372402
    [TF Workshop] Preparing map ID 454372402
    [TF Workshop] Error fetching updated information for map id 454372402
    [TF Workshop] Info lookup failed for workshop file 454372402 ( EResult 1 )
    [TF Workshop] Map failed to sync, load will not go well :(
    CModelLoader::Map_IsValid:  No such map 'maps/workshop/454372402.bsp'
    Unable to change level!
    In gameover, but failed to load the next map. Trying next map in cycle.
    You can also run a Workshop map directly, using either of the two formats above in conjunction with the rcon command changelevel, or the SourceMod map changing commands /map and /setnextmap. When the server tries to switch to that map it will begin to track and download it. You can track a map manually in an attempt to download it prior to switching to it, using the command tf_workshop_map_sync <ugc id>.


    Differences Between Stock and SourceMod

    You can use either of the above formats in your map cycle on a vanilla server, it does not matter, because when the server loads, it will resolve all of the UGC IDs in your mapcycle to their true filenames. So when a player goes to call a map vote, this is what they see:

    [​IMG]

    Note in the screenshot above that the first number is that of a map that is no longer present on the Workshop (a dead map). Also note that the maps have been ordered alphabetically by filename and include version numbers.

    If you use the simplified format in your mapcycle, this is what players will see in a SourceMod map menu:

    [​IMG]

    If you use the second, extended format with friendly name, this is what players will see:

    [​IMG]

    SourceMod is programmed to stop the last few maps from being played again. Unfortunately this doesn't work in the current public versions of SourceMod unless you use the true map filename in place of the friendly name. You can find those by loading up your server and using the console command tf_workshop_map_status to see all of the currently tracked maps.


    Known Issues

    Most of the bugs with Workshop maps have been ironed out but you should keep an eye out for new ones. Reports of problems aren't uncommon each time Valve makes an update.

    There seems to be a bug preventing SourceMod servers from changing Workshop maps, at the moment. I have experienced it locally and have received a report of it in this thread. I will update this when I know more. (5th September 2015)


    FAQ

    :cap: Why are my SourceMod map lists unpopulated?

    There was a problem with SourceMod that was introduced after an update to TF2 where the map handling code stopped adding maps to map list menus. This was fixed in SourceMod snapshot 1.7.3-git5214 and did not occur in 1.8 beta.

    :cap: How can I delete all of my workshop maps and reset the server's tracking record of them?

    Whilst the server is shut down, delete steamapps/workshop/appworkshop_440.acf and everything inside steamapps/workshop/content/440.


    See Also


    Problems, Comments and Suggestions?

    By all means, post in this thread or contact me using the private message system or on Steam. Or pop in to the TF2maps.net Steam chat room if you'd like to chat with mappers and some server operators and developers.
     
    • Thanks Thanks x 14
    Last edited: Sep 4, 2015
  2. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Update:

    I used the contact form on the Team Fortress web site to inquire about the issues above, and am happy to report that I received a response confirming their knowledge of the problem and their assurance that it will be fixed in a future update. The gent said that the problem is with the way the game loads maps.

    So with that in mind, it's probably safe to start hosting workshop maps if you aren't concerned about the problems.

    EDIT: What we need is some kind of tool that compares a list of maps to the workshop and gets their ID numbers.
     
    • Thanks Thanks x 4
    Last edited: Jun 5, 2015
  3. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    Awesome, good job getting the info to them. I think we should compile a list of issues/ feature requests and get it to them soon.
     
  4. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Thanks, Crash. Yes, I think we should have a thread dedicated to discussion about maps on the workshop and am surprised one does not exist.
     
  5. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,742
    Is there a preferred method to get such maps onto a server's fastdl server?

    edit: if the workshop acts as the fastdl, why is it broken for me? :(

    [TF Workshop] Preparing map ID 454165353 for load
    [TF Workshop] DownloadItem failed for file, map will not be usable [ 454165353 / ctf_frosty_b2.bsp ]
    [TF Workshop] Preparing map ID 454165353 for load
    [TF Workshop] Map failed to sync, load will not go well :(
    Missing map maps/workshop/ctf_frosty_b2.ugc454165353.bsp, disconnecting
     
    Last edited: Jun 7, 2015
  6. Berry

    aa Berry Sharing is caring, and I don't care

    Messages:
    985
    Positive Ratings:
    1,483
    You need the steam client beta turned off for this to work for some reason. Not sure if it's meant to fix it, but it did for me.
     
  7. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,742
    Nope. I just went from not in the beta, to in the beta and it worked, so it's more likely to be a steam restart that was needed. My previously empty workshop folder is now full :)

    Tested everything and it was me and my local biz that was the problem :D
     
  8. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    So word is that updating your map in the workshop breaks it. It sounds like the server will update, but the client won't. This is a nasty bug, so I recommend holding off on updating your maps until it's taken care of, hopefully soon.
     
  9. sevin

    aa sevin

    Messages:
    959
    Positive Ratings:
    662
    The workshop maps I've subscribed to have been downloading to tf/maps/workshop. You can't load them from in game unless you copy them out to the maps or download/maps folder though.
     
  10. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    You can load them in the game by using the workshop/ prefix in your map or changelevel commands.

    Crash, I have experimented with updating a workshop map today, using the same filename, and I have different results. I created a map and published it, then did tf_workshop_map_sync {mapid} on an online server I have access to. I joined it using my client, and the map downloaded fine.

    Then I updated the map on the workshop with a modified BSP that had the same file name. When I connected to the online server this time, my client said 'Updating Steam resources...', then I was prevented from connecting and got an error window saying my version of the map differed from the server's. I loaded the map locally, and I saw I had downloaded the new version. I tried to get the online server to download the new version by changing levels, using the tf_workshop_refresh command, and restarting the server, but it still seems to be hosting the old version.

    Regardless of who is worse off, client or server, there are certainly problems, so as you said, it is advisable to not update a workshop map at all for the time being.

    I will be posting a detailed thread about workshop maps for client users and authors, soon.

    Thanks for the info!
     
    Last edited: Jun 8, 2015
  11. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
  12. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    I experimented with uploading an updated map with a different file name and can tell you that, upon connecting to the server whilst it was running the old file name map, my client downloaded the new file name map during the connection process. It then loaded up the old version and I was able to play it on the server. A set of both maps, each with its unique file name, now exist in maps/workshop and SteamApps/workshop/content/440/{mapid}.

    But there still doesn't seem to be a way to get the server to download the new version. Just like the situation before, where the file name had not changed. It should be doing this automatically, according to Valve HLDS. So if an author updates their map, and a server is still hosting the old version, it seems that the client will download a version that is too new for the server and won't be able to play.

    tf_workshop_refresh does not work on the server I was testing with, or my local server. But it does work in my client. The console messages says:
    This leads me to believe that the command is actually for refreshing the list of subscribed maps. Perhaps, if a TF2 server can be associated with a Steam account, then the owner of that account could subscribe to maps and refresh them occasionally.

    I have also learned that if you delete a map from maps/workshop and then use tf_workshop_map_sync {mapid} it will copy the map from SteamApps/workshop/content/440/{mapid} rather than download it again. If you delete both of the BSPs in these locations, the console will say:
    This is the same for both the client and server. Somewhere, even on an anonymous server, it must be keeping a record of which map files it has already downloaded. I have tried deleting all the log and cache files I deem appropriate, but it has had no effect. If you want to get the map back on your server, you need to subscribe to it in the workshop and copy it from your machine to the server's SteamApps/workshop/content/440/{mapid}, then use the tf_workshop_map_sync command again. Subscribing and then immediately unsubscribing is enough to fix the download for your client if you are getting the above error message. So a server operator is advised to back up the maps in the SteamApps dir.

    So try not to delete maps from SteamApps/workshop/content/440/{mapid} because it seems fuck it up.

    I will try to experiment with associating a server with a Steam account to see if it will update subscribed maps.

    EDIT: I created a Steam Web API key, used it to make a Steam game server ID, and got some TF game server login keys associated with my Steam account. I logged in with all of that on my local server, but the server does not respond to my own subscribed maps when I use tf_workshop_refresh. The command seems to do nothing, on a server. Logging in using these new credentials did not affect the supposed automatic updating.

    One thing I did notice, though, is when I tried to do a tf_workshop_map_sync on yet another new file name version of my map, on my local server, it gave me the error I quoted above, but it used the file name of the new BSP.

    So yeah, don't update your workshop maps.
     
    Last edited: Jun 8, 2015
  13. DrLambda

    aa DrLambda L69: Teeheehee, Member

    Messages:
    458
    Positive Ratings:
    454
    If that's true, that's a recent issue, because i was using the workshop version of koth_temper for a LAN party on thursday, and i updated it to b7 on Wednesday. And i'm pretty sure there were a few guys there that subscribed to the map before.
     
  14. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    Got a reply from valve regarding updating maps:

    Edit:
     
    • Thanks Thanks x 3
    Last edited: Jun 9, 2015
  15. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    Oh and a second email regarding whether or not we should iterate our map names (a1/ a2/ etc)!

     
    • Thanks Thanks x 3
    Last edited: Jun 9, 2015
  16. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Lovely stuff. Very valuable information. Thanks, Crash.
     
  17. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    I'd also like to mention that that was like a half hour turn around time from when I sent my original email. Valve is paying attention!
     
  18. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    I take it as a sign they are very excited to be getting some new features done. Or they are giving you preferential treatment ;-) It doesn't look like an automated response. I wonder how much feedback they are receiving about the workshop? I don't go to Reddit or Facepunch so I don't know how much discussion is going on about it elsewhere.

    It's great that these responses are forthcoming.
     
  19. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,742
    if either method works, we should still version. It'll be so much easier for feedback and debugging and everything.

    Once you get to stable releases though, it's debatable if you'd need to, I think I still will.
     
  20. Crash

    aa Crash func_nerd

    Messages:
    3,158
    Positive Ratings:
    4,780
    Yeah I personally plan on continuing to iterate. I can't see the workshop taking over all map sharing, so it would still cause issues.