Compile time > 1hr

Discussion in 'Mapping Questions & Discussion' started by bamibal, Sep 30, 2015.

  1. bamibal

    bamibal L1: Registered

    Messages:
    47
    Positive Ratings:
    5
    Doing a full compile of my map takes over an hour:( I guess that's way too long, even for my slightly large map with many lights. Optimizations I made were making lots of non-structural brushes func_detail and applying nodraw to many faces that a player will never get to see.

    I need to add some cubemaps and rebuild them, so I have to do a new full compile. Sucks to wait an hour again, plus now I wonder if my map isn't too poorly optimized to perform well for users. Correct me if I'm wrong but isn't the last thing I can do to optimize is to build areaportals? Will this bring my compile time or is it only profitable for players? If it does help, can anyone help me understand how to use them, because no matter how many tutorials I read on them, I still don't understand the principle.
     
  2. Snowbat

    Snowbat L4: Comfortable Member

    Messages:
    165
    Positive Ratings:
    61
    You don't have to do a full compile every time you want to test your map. You can do a fast compile which takes significantly less time.
    I found that most of the time when my compile time skyrockets, it's either because I forgot to turn complex brushwork into func_detail, or because I have really big open rooms in my map.
    Areaportals are a powerful tool if used properly. They can decrease compile times and optimize, but agian, you need to know what you're doing.

    Perhaps post a compile log and a couple of screenshots of your main areas of your map?
     
    • Thanks Thanks x 1
  3. killohurtz

    aa killohurtz Distinction in Applied Carving

    Messages:
    1,003
    Positive Ratings:
    1,155
    First thing to check is whether you have leaks.

    Second, check your visleaf complexity. After VBSP has run, it will generate a portal file, which you can then load in Hammer by clicking Map > Load Portal File (you can run VBSP only and check "Don't run the game" from the Run Map dialog to update the portal file quickly). You should then see blue lines all over your map - these represent the edges of your visleaves. Fly around in the 3D view, look for areas with a high density of blue lines, and clean them up by func_detailing things or adding hint faces.

    Other things to check:
    • No lights are named unless you are moving them/turning them off or on
    • Reasonable lightmap scales
     
    • Thanks Thanks x 1
  4. Bunbun

    aa Bunbun Gay sex slave

    Messages:
    403
    Positive Ratings:
    569
    Is it portal flow that slows you down?
     
    • Thanks Thanks x 1
  5. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Area portals benefit the people playing your map, as they calculate what is visible through them. I don't think they can do anything to improve compile time, only rendering performance.

    If all you want to do is add some point entities to your map, you can do an entity-only compile (see -onlyents). This will give you a message in TF2's console that the map's ZIP is stale, but you might be able to get rid of that by removing a text file from the map's pak that's named something like stalezip.txt. Entity-only compiles only need VBSP, so disable the VVIS and VRAD portions of your compile.

    As Snowbat says, you can perform a fast compile, or just a VBSP-only compile, and load the generated portal file in Hammer (Map > Load Portal File) to see which brushes are creating unwanted visleaves. Further visleaf improvements will help reduce the VVIS portion of your compile, but if a large amount of time is given to VRAD, then the only cure for that is a better computer.

    Post your VMF if you like.
     
    • Thanks Thanks x 1
  6. bamibal

    bamibal L1: Registered

    Messages:
    47
    Positive Ratings:
    5
    Snowbat, from what I read a new full compile removes old cubemap data. I need to rebuild my cubemaps so I need the old ones to be deleted (I read that rebuilding cubemaps on top of old ones will just increase filesize). Or can I build cubemaps on a fast compile too?

    Here's the log of a quick compile (don't know if it's usable? else I have to do a full one again)

    ** Executing...
    ** Command: "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\bin\vbsp.exe"
    ** Parameters: -game "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\tf" "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.vmf"

    Valve Software - vbsp.exe (Sep 22 2015)
    4 threads
    materialPath: C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\tf\materials
    Loading C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.vmf
    Patching WVT material: maps/abandoned_prison/swamp/nature/blendswampmudtodirt001_wvt_patch
    fixing up env_cubemap materials on brush sides...
    ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (1)
    ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (1)
    WARNING: node without a volume
    WARNING: node without a volume
    Processing areas...done (0)
    Building Faces...done (1)
    Chop Details...done (0)
    Find Visible Detail Sides...
    Merged 622 detail faces...done (0)
    Merging details...done (0)
    FixTjuncs...
    PruneNodes...
    WriteBSP...
    done (1)
    writing C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.prt...Building visibility clusters...
    done (0)
    *** Error: Skybox vtf files for skybox/sky_night_01 weren't compiled with the same size texture and/or same flags!
    Can't load skybox file skybox/sky_night_01 to build the default cubemap!
    *** Error: Skybox vtf files for skybox/sky_night_01 weren't compiled with the same size texture and/or same flags!
    Can't load skybox file skybox/sky_night_01 to build the default cubemap!
    Finding displacement neighbors...
    Found a displacement edge abutting multiple other edges.
    Finding lightmap sample positions...
    Displacement Alpha : 0...1...2...3...4...5...6...7...8...9...10
    Building Physics collision data...
    WARNING: Map using power 4 displacements, terrain physics cannot be compressed, map will need additional memory and CPU.
    done (1) (2447700 bytes)
    Placing detail props : 0...1...2...3...4...5...6...7...8...9...10
    Compacting texture/material tables...
    Reduced 4575 texinfos to 2238
    Reduced 395 texdatas to 347 (19665 bytes to 17279)
    Writing C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp
    15 seconds elapsed

    ** Executing...
    ** Command: "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\bin\vvis.exe"
    ** Parameters: -game "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\tf" -fast "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison"

    Valve Software - vvis.exe (Sep 22 2015)
    fastvis = true
    4 threads
    reading c:\program files (x86)\steam\steamapps\common\team fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp
    reading c:\program files (x86)\steam\steamapps\common\team fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.prt
    4412 portalclusters
    13154 numportals
    BasePortalVis: 0...1...2...3...4...5...6...7...8...9...10 (7)
    Optimized: 1226082 visible clusters (11.21%)
    Total clusters visible: 10932585
    Average clusters visible: 2477
    Building PAS...
    Average clusters audible: 4330
    visdatasize:4134213 compressed from 4870848
    writing c:\program files (x86)\steam\steamapps\common\team fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp
    8 seconds elapsed

    ** Executing...
    ** Command: "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\bin\vrad.exe"
    ** Parameters: -game "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\tf" -noextra "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison"

    Valve Software - vrad.exe SSE (Sep 22 2015)

    Valve Radiosity Simulator
    4 threads
    [Reading texlights from 'lights.rad']
    unknown light specifier type - lights

    [56 texlights parsed from 'lights.rad']

    Loading c:\program files (x86)\steam\steamapps\common\team fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp
    Setting up ray-trace acceleration structure... Done (6.95 seconds)
    18850 faces
    16 degenerate faces
    871727 square feet [125528800.00 square inches]
    92 Displacements
    6858 Square Feet [987561.75 Square Inches]
    18834 patches before subdivision
    108456 patches after subdivision
    sun extent from map=0.087156
    243 direct lights
    BuildFacelights: 0...1...2...3...4...5...6...7...8...9...10 (9)
    BuildVisLeafs: 0...1...2...3...4...5...6...7...8...9...10 (62)
    transfers 14139743, max 1303
    transfer lists: 107.9 megs
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #1 added RGB(117774, 89898, 61112)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #2 added RGB(43555, 27977, 15842)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #3 added RGB(17294, 9428, 4553)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #4 added RGB(6968, 3239, 1343)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #5 added RGB(2850, 1143, 415)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (1)
    Bounce #6 added RGB(1180, 414, 133)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #7 added RGB(495, 155, 45)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #8 added RGB(211, 61, 16)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #9 added RGB(92, 25, 6)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #10 added RGB(41, 11, 3)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #11 added RGB(19, 5, 1)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #12 added RGB(9, 3, 0)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #13 added RGB(4, 1, 0)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #14 added RGB(2, 1, 0)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #15 added RGB(1, 0, 0)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #16 added RGB(1, 0, 0)
    Build Patch/Sample Hash Table(s).....Done<0.0299 sec>
    FinalLightFace: 0...1...2...3...4...5...6...7...8...9...10 (15)
    FinalLightFace Done
    0 of 0 (0% of) surface lights went in leaf ambient cubes.
    ThreadComputeLeafAmbient: 0...1...2...3...4...5...6...7...8...9...10 (11)
    Writing leaf ambient...done
    Ready to Finish

    Object names Objects/Maxobjs Memory / Maxmem Fullness
    ------------ --------------- --------------- --------
    models 139/1024 6672/49152 (13.6%)
    brushes 5136/8192 61632/98304 (62.7%)
    brushsides 42279/65536 338232/524288 (64.5%)
    planes 22902/65536 458040/1310720 (34.9%)
    vertexes 34169/65536 410028/786432 (52.1%)
    nodes 10714/65536 342848/2097152 (16.3%)
    texinfos 2238/12288 161136/884736 (18.2%)
    texdata 347/2048 11104/65536 (16.9%)
    dispinfos 92/0 16192/0 ( 0.0%)
    disp_verts 3572/0 71440/0 ( 0.0%)
    disp_tris 5152/0 10304/0 ( 0.0%)
    disp_lmsamples 21285/0 21285/0 ( 0.0%)
    faces 18850/65536 1055600/3670016 (28.8%)
    hdr faces 0/65536 0/3670016 ( 0.0%)
    origfaces 12317/65536 689752/3670016 (18.8%)
    leaves 10854/65536 347328/2097152 (16.6%)
    leaffaces 21274/65536 42548/131072 (32.5%)
    leafbrushes 14656/65536 29312/131072 (22.4%)
    areas 2/256 16/2048 ( 0.8%)
    surfedges 137812/512000 551248/2048000 (26.9%)
    edges 82063/256000 328252/1024000 (32.1%)
    LDR worldlights 243/8192 21384/720896 ( 3.0%)
    HDR worldlights 0/8192 0/720896 ( 0.0%)
    leafwaterdata 0/32768 0/393216 ( 0.0%)
    waterstrips 2047/32768 20470/327680 ( 6.2%)
    waterverts 0/65536 0/786432 ( 0.0%)
    waterindices 32922/65536 65844/131072 (50.2%)
    cubemapsamples 119/1024 1904/16384 (11.6%)
    overlays 76/512 26752/180224 (14.8%)
    LDR lightdata [variable] 4312272/0 ( 0.0%)
    HDR lightdata [variable] 0/0 ( 0.0%)
    visdata [variable] 4134213/16777216 (24.6%)
    entdata [variable] 399428/393216 (101.6%) VERY FULL!
    LDR ambient table 10854/65536 43416/262144 (16.6%)
    HDR ambient table 10854/65536 43416/262144 (16.6%)
    LDR leaf ambient 29929/65536 838012/1835008 (45.7%)
    HDR leaf ambient 10854/65536 303912/1835008 (16.6%)
    occluders 0/0 0/0 ( 0.0%)
    occluder polygons 0/0 0/0 ( 0.0%)
    occluder vert ind 0/0 0/0 ( 0.0%)
    detail props [variable] 1/12 ( 8.3%)
    static props [variable] 1/117716 ( 0.0%)
    pakfile [variable] 103096/0 ( 0.0%)
    physics [variable] 2447700/4194304 (58.4%)
    physics terrain [variable] 0/1048576 ( 0.0%)

    Level flags = 0

    Total triangle count: 52214
    Writing c:\program files (x86)\steam\steamapps\common\team fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp
    1 minute, 47 seconds elapsed

    ** Executing...
    ** Command: Copy File
    ** Parameters: "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\sourcesdk_content\tf\mapsrc\Abandoned_Prison.bsp" "C:\Program Files (x86)\Steam\steamapps\common\Team Fortress 2\tf\maps\Abandoned_Prison.bsp"

    And some screenshots

    [​IMG]
    Top view of the map

    [​IMG]
    There are three of these wings

    [​IMG]
    And numerous rooms like this (looked at from the outside through nodraw faces)
     
  7. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Wow, those screenshots look good.

    A new compile will overwrite your BSP file, which means everything is lost.

    If you go through the process of building cubemaps again, the existing VTFs will be overwritten. If you change the map's filename after you have already built cubemaps for it, then you will end up with two sets of cubemaps, as they are stored inside the BSP in a folder that has the same name as the map.

    You can build cubemaps for any BSP regardless of the compile, as long as it has env_cubemap entities inside it.

    Note that once a compile finishes, the BSP is copied to the game directory. Normally, the BSP lives in the same directory as your VMF. When you build cubemaps, the copy is the file that receives them, while your source BSP is not touched again. If you disable the 'Copy' step of the compile, then the BSP that lives in your game's maps directory will not be overwritten.
     
    • Thanks Thanks x 1
  8. bamibal

    bamibal L1: Registered

    Messages:
    47
    Positive Ratings:
    5
    Fist off, thanks for the many informative answers! I'm slowly working through them and trying things. I'm not nearly experienced enough to understand everything immediately so it takes some time. Thanks for the patience.

    My portal file shows a clusterf*ck of lines on top of the many arches I have. Is this something I have to be worried about? This is on the outside of the roof and won't be seen by players (hence the nodraw).

    [​IMG]
     
  9. Hyperion

    aa Hyperion L16: Grid Member

    Messages:
    810
    Positive Ratings:
    617
    • Thanks Thanks x 2
  10. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,032
    Positive Ratings:
    3,979
    You could actually just throw some hint brushes down, you don't need to make those square.

    [​IMG]

    This will make those visleafs a lot easier to figure out and much faster in your compile.

    Here's an example of some brushwork I hinted rather than func_detailing (I hid all the func_details):

    [​IMG]

    In this case, it works just as well if not better to hint along the outer white planks than to func_detail them, because this actually makes visleafs less complex and faster to make during compile. If I f_d these planks, I'd still have to hint here anyway.

    Long hallway with windows:

    [​IMG]

    These could be areaportaled, but anyone in the area will see in here anyway, so each areaportal's visibility would need to be calculated by the client. It's an example of a situation where putting areaportals in every door/window is a bad idea.

    [​IMG]

    Similar thing here. I don't need the doors/windows of that angled building to have areaportals. I also want the deck to not cut visleafs out in the open area (no reason to).

    Hope this helps. Simple hints are good hints.

    The only reason to do this is if players will be above the arch or will see where it meets a wall (like 2fort sewers, arched entrances) or there is geometry above it that the arches should meet (so you avoid doing VVIS for places no one will see or use).
     
    • Thanks Thanks x 4
    Last edited: Sep 30, 2015
  11. ibex

    aa ibex

    Messages:
    255
    Positive Ratings:
    368
    Yes. Honestly, I func_detail almost every arch I make. In your case since these arches are above the players and don't need to be cutting any sightlines/visleaves, then you should func_detail them. Also, I can't particularly tell what's going on, but are you compiling in a box?
     
    • Thanks Thanks x 1
  12. killohurtz

    aa killohurtz Distinction in Applied Carving

    Messages:
    1,003
    Positive Ratings:
    1,155
    Did you surround that entire building in skybox brushes? If players are never allowed out of the building, then there is a much simpler way to pull off a skybox that eliminates unnecessary visleaves outside your map:
    1. Place skybox brushes just big enough to cover each window that you want to see the sky through. Think of this as just patching holes.
    2. Make sure the rest of the building is sealed.
    3. Compile! Now that the exterior is touching the void, no visleaves will be generated outside, and as a bonus, every face on the outside is automatically culled as if you nodrawed it.
     
    • Thanks Thanks x 2
  13. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,032
    Positive Ratings:
    3,979
    Mm, yeah, killohurtz makes a great point. Looking at your other screenshots, it looks like you should either have skybox around those anyway (or nothing if they aren't going to be func_detail). If your map is sealed right, VVIS won't calculate in the void anyway, like he says.

    Arches do not always need to be func_detailed, and this is actually a case where doing so would be worse. There's nothing inherently wrong with weird shaped visleafs.
     
    • Thanks Thanks x 1
  14. bamibal

    bamibal L1: Registered

    Messages:
    47
    Positive Ratings:
    5
    Damn, how could I not think of this! I'm going to redo the skyboxes and report back.
     
  15. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Hi, bamibal.

    No worries. nodraw is one thing, but it is not the start and end of optimisation. It helps only rendering performance and lightmap file size, by reducing the number of faces that need to be drawn.

    The problem you have here is that the outside of the brushes in your screenshot should not be 'inside' the map's space and therefore cutting visleaves. They should be exposed to the void, the area outside the map. Visibility is not calculated for things that are outside the bounds of the map. In the screenshot, it looks like you have enclosed these brushes in a box made of brushes, probably nodraw or skybox. But you do not need to do that. The brushes on their own are enough to seal the map. Try taking the enclosing box out, do a VBSP-only compile and load the portal file again. You can hide things by selecting them and pressing H. Make sure you save the map before you compile if you hide stuff, because the compile uses the saved VMF from the hard disk, and not the map resident in Hammer's memory.

    As you can see in this example archwork, the visleaf is drawn cleanly inside because it is a concave shape:

    [​IMG]

    But if you take a look outside, with the portal file loaded, there are no portals because the space is outside the map. Even though the shape is a complex, convex archway, it does not matter because it's facing the void.

    [​IMG]

    A note about textures in the void: It's safe to leave void-facing brush faces textured, because they are automatically culled by VBSP when it creates the BSP. You do not need to nodraw the outside of your map. In fact, nodraw is something you do not need to do proactively because most of the time, an unseen face will be culled. It's only when you have compiled your map (can be a fast compile) and flown around it in game should you look for faces to nodraw. Faces that have no chance of being seen by the player.

    This thread is also good reading.
     
    • Thanks Thanks x 1
  16. ibex

    aa ibex

    Messages:
    255
    Positive Ratings:
    368
    Could you explain why it would be worse?
     
    • Thanks Thanks x 1
  17. killohurtz

    aa killohurtz Distinction in Applied Carving

    Messages:
    1,003
    Positive Ratings:
    1,155
    VBSP is capable of generating visleaves that fit nicely under arches (see worMatty's post), especially if you use hints to help it. In bamibal's map, the arches are helping to seal the building, so if he's going to forgo a complete skybox then the building cannot have func_detailed arches.
     
    • Thanks Thanks x 3
  18. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    See my post above. But for more reasons:
    • func_detail is lit differently to world brushes, and they have harsher shadowing. When a func_detail is placed on a world brush, the shadow it casts on to the world brush is not as realistic as if the object was a world brush. Lightmaps on func_detail faces that are aligned with world brush faces can be different, so they stand out and look odd.
    • It is more efficient from a work perspective to use VBSP culling to one's advantage in this case, than to select all the arches, make them func_detail, texture the hidden faces with nodraw and surround them in a sealing box. The sealing box could adversely affect visibility optimisation as it spans multiple rooms in bamibal's map. Leaving the arches as world brushes is a single action that does not need to be countered because visibility and textured face culling will be handled automatically.
    These days I don't func_detail anything unless it's fiddly and obviously does not block player vision, such as thin, narrow platforms or free-standing posts. Leaving things as world brushes is more efficient. When I'm ready to compile, I do a VBSP-only compile and check the portal file to see if the visleaves need refining, like in tyler's example screenshots. But even if something small cuts visleaves it's sometimes more beneficial to leave it as a world brush because of the lighting and shading, and that more visleaves isn't necessarily a problem. It's the pointless, small visleaves that can be merged/shaped/combined if the effort is worth it.
     
    • Thanks Thanks x 4
    Last edited: Sep 30, 2015
  19. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,032
    Positive Ratings:
    3,979
    Because then the top of these areas would be open to the void, and then you'd have to add more brushes to seal it again. As Matty and Killo have explained, it's best to leave those arches exposed to the void anyway, where VVIS won't be calculated. If they create problematic visleaf structures inside the map, they can be adjusted with a single hint. As func_details, they may also create subtle lighting issues. It's a lot better than creating a nodraw frame around the func_detail archway. In a large finished map, those few extra brushes can make the difference.
     
    • Thanks Thanks x 2
  20. bamibal

    bamibal L1: Registered

    Messages:
    47
    Positive Ratings:
    5
    A harsh reminder of why I had decided to do the 'skybox around everything'; every single one of the +/- 90 arched roofs in my map leaked:confused: Apparently the first one I made went off-grid and then I copy-pasted it 90 times throughout the map. Spent the last couple of hours getting them back on the grid. Ah well, I guess it's not unusual to make these stupid mistakes in a first map:p

    But the good news is that my map just compiled in under 5 minutes. Crazy, from over an hour to under 5 minutes just by removing the large skyboxes. Thanks a bunch for all the advice!