Obscene compile time

Discussion in 'Mapping Questions & Discussion' started by Scampi, Jan 2, 2018.

  1. Scampi

    Scampi L4: Comfortable Member

    Messages:
    158
    Positive Ratings:
    104
    MY MAP HAS BEEN COMPILING FOR OVER A WEEK. I AM SCARED AND CONFUSED.

    It's still progressing through vvis, so I know it's doing something. I'm really quite interested as to why this is taking such an absolutely absurd length of time. The map is a zeppelin made from displacements with an interior floating in a massive empty skybox. I'm wondering what's causing this, as I've never heard of something like this happening before. The map does not use func_details (it probably should), but I've compiled larger and more detailed maps than this without them and it never took anywhere close to this length. Here's some info about my map, anything stand out as obviously problematic?

    - Lots of props and custom content
    - Skybox around ship is as large as possible (TF2-themed gmod machinima map)
    - Zeppelin exterior made largely from displacements with some small props and brushes
    - Detailed interior with lights, overlays, etc. Made from brushes.
    - Nodraw on every single nonvisible surface
    - Full final compile (fast compiles take seconds)

    Should I have cut my losses early on? Yes. Am I too invested to back out now? That's certainly... a question.
     
  2. Vel0city

    aa Vel0city func_fish

    Messages:
    1,938
    Positive Ratings:
    1,540
    A week? I'm pretty sure it's just stuck and not doing anything at this point.

    Anyway, you saying it's stuck on VVIS makes me think you haven't used a lot of func_detail, if any, and judging by your description of your map it has very complex geometry. Use func_detail to tell VVIS to not use brushes for visibility calculations that can't block visibility in any way such as brushes used for fine details. Read more on optimisation in the optimisation bible here: http://www.optimization.interlopers.net .
     
  3. RaVaGe

    aa RaVaGe

    Messages:
    728
    Positive Ratings:
    1,055
    • Agree Agree x 2
    • Thanks Thanks x 1
  4. Lampenpam

    aa Lampenpam

    Messages:
    1,019
    Positive Ratings:
    337
    if you have large open spaces, it takes hours to compile vvis for it even if the actual map is optimized. (it is optimized for vvis, isn't it?) turning the entire open space into singel giant vis leafs can reduce compile times dramaticly. Mind that while being inside the viscluster a lot of other stuff from the map might be rendered so it's better to only have the vis_viscluster cover a volume the players are rarely in or are about to fall into a death pit anyway.
     
    • Thanks Thanks x 1
  5. Crash

    aa Crash func_nerd

    Messages:
    3,191
    Positive Ratings:
    4,924
    func_detailing all your smaller details will help a ton, but the large open space isn't doing you any favors either.

    REALLY important to learn how to properly use func_details. Here's a good guide to optimization that will explain func_details, too.
     
    • Agree Agree x 1
  6. henke37

    aa henke37

    Messages:
    1,943
    Positive Ratings:
    460
    At this point it might be a good idea to show the map layout. Or even better, just share the vmf file.
     
  7. Lampenpam

    aa Lampenpam

    Messages:
    1,019
    Positive Ratings:
    337
    first he should use the viscluster which will most likekly do the job as long he knows how to optimize the rest.
     
    • Respectfully Disagree Respectfully Disagree x 2
    Last edited: Jan 2, 2018
  8. Crash

    aa Crash func_nerd

    Messages:
    3,191
    Positive Ratings:
    4,924
    They are saying the map has zero func_details and a bunch of small details in brushwork. Start with that, if it's not helping as much as needed, then try visclusters.

    It's REALLY easy to break optimization on your map with visclusters.
     
  9. Scampi

    Scampi L4: Comfortable Member

    Messages:
    158
    Positive Ratings:
    104
    Alright, compile aborted. Yeah, I figured func_details would help. Funny though, I've recompiled a version of mann manor with all func_details turned converted to normal brushes and it took me a day. I predict this step will be one of several. Time to test...
     
  10. Lampenpam

    aa Lampenpam

    Messages:
    1,019
    Positive Ratings:
    337
    woobs, I skipped the part where he said he didn't use any func_detail. Well, visclusters are still required for his map if he has a massive open space or func_detail alone will only solve half of the problem.
     
  11. BigfootBeto

    aa BigfootBeto Too obsessed with perfection

    Messages:
    403
    Positive Ratings:
    599
    For reference, vvis should rarely take longer than 15 minutes for a complex map (and if it's a simple map like an alpha map, then it should only take a few seconds). vrad is what should be the bulk of compile time.
     
    • Respectfully Disagree Respectfully Disagree x 1
  12. Scampi

    Scampi L4: Comfortable Member

    Messages:
    158
    Positive Ratings:
    104
    Thank you gentlemen, all is well. I did some testing and visclusters made an incredible difference. The func_detailing no doubt helped as well.

    *edit*

    The compile appeared successful... But there are a few issues. This map is a TF2 themed machinima map, so I care more about how it runs in gmod than in TF2 itself. However both TF2 and gmod have issues with the map. In TF2, the skybox is missing, resulting in that glorious neon checkerboard. However, it looks perfect otherwise. In gmod, the map is stuck in fullbright. Here's the compile log:

    Starting a 'Publish' compile.
    Starting compilation of gm_zeppelin
    Valve Software - vbsp.exe (Oct 14 2017)
    8 threads
    materialPath: C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf\materials
    Loading C:\Users\name\Desktop\HAMMER\gm_zeppelin.vmf
    fixing up env_cubemap materials on brush sides...
    ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (0)
    ProcessBlock_Thread: 0...1...2...3...4...5...6...7...8...9...10 (1)
    Processing areas...done (0)
    Building Faces...done (0)
    Chop Details...done (0)
    Find Visible Detail Sides...
    Merged 352 detail faces...done (0)
    Merging details...done (0)
    FixTjuncs...
    PruneNodes...
    WriteBSP...
    done (0)
    writing C:\Users\name\Desktop\HAMMER\gm_zeppelin.prt...Building visibility clusters...
    done (0)
    Creating default LDR cubemaps for env_cubemap using skybox materials:
    skybox/sky05*.vmt
    ! Run buildcubemaps in the engine to get the correct cube maps.
    Creating default HDR cubemaps for env_cubemap using skybox materials:
    skybox/sky05*.vmt
    ! Run buildcubemaps in the engine to get the correct cube maps.
    Finding displacement neighbors...
    Found a displacement edge abutting multiple other edges.
    Finding lightmap sample positions...
    Displacement Alpha : 0...1...2...3...4...5...6...7...8...9...10
    Building Physics collision data...
    done (0) (299636 bytes)
    Placing detail props : 0...1...2...3...4...5...6...7...8...9...10
    Compacting texture/material tables...
    Reduced 2216 texinfos to 1260
    Reduced 39 texdatas to 37 (804 bytes to 715)
    Writing C:\Users\gcamp\Desktop\HAMMER\gm_zeppelin.bsp
    Wrote ZIP buffer, estimated size 165061, actual size 164311
    4 seconds elapsed
    Valve Software - vvis.exe (Oct 14 2017)
    8 threads
    reading c:\users\name\desktop\hammer\gm_zeppelin.bsp
    reading c:\users\name\desktop\hammer\gm_zeppelin.prt
    604 portalclusters
    2068 numportals
    BasePortalVis: 0...1...2...3...4...5...6...7...8...9...10 (0)
    PortalFlow: 0...1...2...3...4...5...6...7...8...9...10 (583)
    Optimized: 685 visible clusters (0.52%)
    Total clusters visible: 130865
    Average clusters visible: 216
    Building PAS...
    Average clusters audible: 555
    visdatasize:89820 compressed from 96640
    writing c:\users\name\desktop\hammer\gm_zeppelin.bsp
    9 minutes, 43 seconds elapsed
    Valve Software - vrad.exe SSE (Oct 14 2017)

    Valve Radiosity Simulator
    8 threads
    [Reading texlights from 'lights.rad']
    unknown light specifier type - lights

    [56 texlights parsed from 'lights.rad']

    Loading c:\users\name\desktop\hammer\gm_zeppelin.bsp
    Setting up ray-trace acceleration structure... Done (9.07 seconds)
    4900 faces
    3 degenerate faces
    42720644 square feet [6151772672.00 square inches]
    224 Displacements
    272499 Square Feet [39239856.00 Square Inches]
    4897 patches before subdivision
    zero area child patch
    28283 patches after subdivision
    sun extent from map=0.000000
    68 direct lights
    BuildFacelights: 0...1...2...3...4...5...6...7...8...9...10 (80)
    BuildVisLeafs: 0...1...2...3...4...5...6...7...8...9...10 (12)
    transfers 1204848, max 390
    transfer lists: 9.2 megs
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #1 added RGB(110271, 102476, 112394)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #2 added RGB(14764, 12884, 12356)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #3 added RGB(3070, 2740, 2668)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #4 added RGB(783, 808, 894)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #5 added RGB(249, 305, 403)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #6 added RGB(85, 126, 193)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #7 added RGB(32, 56, 99)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #8 added RGB(12, 25, 50)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #9 added RGB(5, 11, 26)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #10 added RGB(2, 5, 13)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #11 added RGB(1, 2, 7)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #12 added RGB(0, 1, 4)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #13 added RGB(0, 0, 2)
    GatherLight: 0...1...2...3...4...5...6...7...8...9...10 (0)
    Bounce #14 added RGB(0, 0, 1)
    Build Patch/Sample Hash Table(s).....Done<0.0250 sec>
    FinalLightFace: 0...1...2...3...4...5...6...7...8...9...10 (3)
    FinalLightFace Done
    0 of 0 (0% of) surface lights went in leaf ambient cubes.
    ThreadComputeLeafAmbient: 0...1...2...3...4...5...6...7...8...9...10 (12)
    Writing leaf ambient...done
    Computing static prop lighting : 0ThreadComputeStaticPropLighting: 0...1...2...3...4...5...6...7...8...9...10 (5)

    Ready to Finish

    Object names Objects/Maxobjs Memory / Maxmem Fullness
    ------------ --------------- --------------- --------
    models 1/1024 48/49152 ( 0.1%)
    brushes 846/8192 10152/98304 (10.3%)
    brushsides 6703/65536 53624/524288 (10.2%)
    planes 3430/65536 68600/1310720 ( 5.2%)
    vertexes 7765/65536 93180/786432 (11.8%)
    nodes 4612/65536 147584/2097152 ( 7.0%)
    texinfos 1260/12288 90720/884736 (10.3%)
    texdata 37/2048 1184/65536 ( 1.8%)
    dispinfos 224/0 39424/0 ( 0.0%)
    disp_verts 5600/0 112000/0 ( 0.0%)
    disp_tris 7168/0 14336/0 ( 0.0%)
    disp_lmsamples 1035364/0 1035364/0 ( 0.0%)
    faces 4900/65536 274400/3670016 ( 7.5%)
    hdr faces 4900/65536 274400/3670016 ( 7.5%)
    origfaces 1954/65536 109424/3670016 ( 3.0%)
    leaves 4614/65536 147648/2097152 ( 7.0%)
    leaffaces 5496/65536 10992/131072 ( 8.4%)
    leafbrushes 4221/65536 8442/131072 ( 6.4%)
    areas 2/256 16/2048 ( 0.8%)
    surfedges 30816/512000 123264/2048000 ( 6.0%)
    edges 19148/256000 76592/1024000 ( 7.5%)
    LDR worldlights 0/8192 0/720896 ( 0.0%)
    HDR worldlights 68/8192 5984/720896 ( 0.8%)
    leafwaterdata 0/32768 0/393216 ( 0.0%)
    waterstrips 587/32768 5870/327680 ( 1.8%)
    waterverts 0/65536 0/786432 ( 0.0%)
    waterindices 11118/65536 22236/131072 (17.0%)
    cubemapsamples 0/1024 0/16384 ( 0.0%)
    overlays 31/512 10912/180224 ( 6.1%)
    LDR lightdata [variable] 0/0 ( 0.0%)
    HDR lightdata [variable] 1563744/0 ( 0.0%)
    visdata [variable] 89820/16777216 ( 0.5%)
    entdata [variable] 43710/393216 (11.1%)
    LDR ambient table 4614/65536 18456/262144 ( 7.0%)
    HDR ambient table 4614/65536 18456/262144 ( 7.0%)
    LDR leaf ambient 4614/65536 129192/1835008 ( 7.0%)
    HDR leaf ambient 11146/65536 312088/1835008 (17.0%)
    occluders 0/0 0/0 ( 0.0%)
    occluder polygons 0/0 0/0 ( 0.0%)
    occluder vert ind 0/0 0/0 ( 0.0%)
    detail props [variable] 1/12 ( 8.3%)
    static props [variable] 1/37074 ( 0.0%)
    pakfile [variable] 1595211/0 ( 0.0%)
    physics [variable] 299636/4194304 ( 7.1%)
    physics terrain [variable] 50486/1048576 ( 4.8%)

    Level flags = 2

    Total triangle count: 13184
    Writing c:\users\name\desktop\hammer\gm_zeppelin.bsp
    2 minutes, 2 seconds elapsed
    C:\Users\name\Desktop\HAMMER\gm_zeppelin.bsp -> C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf\maps\gm_zeppelin.bsp
    1 File(s) copied
    CompilePal - Nav Generator
    Generating...
    nav file complete!

    CompilePal - Cubemap Generator
    Detecting HDR levels...
    Map requires one set of cubemaps
    Compiling cubemaps...
    Cubemaps compiled

    CompilePal - Automated Packaging
    Finding sources of game content...
    Found search paths...
    Found wildcard path: C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf\custom\
    Found search path: C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf
    Found search path: C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf\bin
    Found search path: C:\program files (x86)\steam\steamapps\common\Team Fortress 2\tf\download
    Reading BSP...
    Initializing pak file...
    Writing file list...
    Running bspzip...
    Finished!
    ---------------------
    37 materials found
    27 models found
    0 particle files found
    0 sounds found
    additional files:
    -nav file
    ---------------------
    'Publish' compile finished in 00:15:11
    2 errors/warnings logged:
    2 errors/warnings logged for gm_zeppelin:
    ● 1x: Info: Found a displacement edge abutting multiple other edges
    ● 1x: Info: zero area child patch
     
    Last edited: Jan 2, 2018
  13. Vel0city

    aa Vel0city func_fish

    Messages:
    1,938
    Positive Ratings:
    1,540
    If the skybox is missing in TF2, then either the skybox material isn't present in TF2's directories or you have called upon the wrong skybox name in the map properties in Hammer.

    For the fullbright in GMOD: check to see if GMOD forces HDR (High Dynamic Range) lighting. If you only compiled with LDR (Low Dynamic Range which it seems you did) then GMOD displays the map in fullbright since it can't find the HDR lighting data. If GMOD does want HDR lighting, compile the entire map again with both LDR and HDR lighting. This will take up more compile time since VRAD has to be run twice in its entirety.
     
    • Agree Agree x 1