[GUIDE] Optimisation, 2020s style

[Tiftid]

I'm generally hesitant to write about topics that others have already covered.

But it seems that I continually see maps get the same things wrong, especially in regard to optimisation.
So this guide will aim to provide a wide breadth of knowledge about intermediate-advanced level optimisation, and a few special tips I've picked up over the years.

1) Remember, no hint brushes​

In my usual fashion, let's start with something that will likely be controversial.
Hint brushes, and func_detail, are generally poorly understood, and recommended as magical solutions to what is really a non-existent problem.

All that hint brushes do is cut visleaves.
It was discovered very early on in Source-engine mapping that there are two scenarios in which this is exceptionally helpful:

Since the way visleaves test visibility is by checking if ANY point within one visleaf can see ANY point within another, this configuration prevents visleaf 1 from seeing any part of visleaf 3, whereas not having this hint brush would have had visleaf 3 slightly stick around the corner and be visible from visleaf 1.

Similar thing here - the vertical hint brush prevents visleaves 1 and 2 from extending above the tops of the buildings, therefore preventing any part of them from seeing each other.

However, these two scenarios are really the only ones where hint brushes will dramatically improve your map's performance, and in most cases where these scenarios occur the performance improvement actually isn't very dramatic at all.

func_detail is a similar case. It's often prescribed to make your compiles faster, since func_detail geometry cannot split visleaves.
But, in practice, non-func-detailed geometry often isn't actually what's making your VVIS compiles so slow.

Here's an example - I can compile cp_bruhstbowl's VVIS on final in under 5 minutes, but back when I was helping to optimise cp_carribean - a much smaller map with far fewer visleaves - VVIS took nearly an hour on the exact same computer.
What's going on?
Well, VVIS has some neat little optimisations it can do.
I'm no programmer, but the way I assume it works is that it sorts all other visleaves in the map front-to-back from the current visleaf, and then tests visibility.
So, if it finds a visleaf that isn't visible, it can preassume in certain cases that all the visleaves behind it also won't be visible.
What this means in practice is that if your map has a lot of visleaves, but most of them are separated from each other by solid geometry, your compile will be lightning-fast.
But, if you have even a small-ish number of visleaves, but a very open map where almost every visleaf can see almost every other one, your compile will take ages and your map will also perform poorly in-game.

Why, then, is my map cp_bruhstbowl so fast to compile, when that map seems to be so open?
It's because of well-partitioned geometry:

The skybox brushes, areaportals, buildings and nodraw brushes underneath the displacements all together form a great dividing wall that TOTALLY restricts visibility from this part of the map to any other, except through the areaportals.
The map has one GIGANTIC skybox ceiling brush, and nearly every building has a skybox brush on top of it that extends all the way up to this ceiling.
This makes it fast and easy to partition off areas of the map.
People generally recommend against using one massive ceiling brush, since it's supposed to crash VBSP, but having other skybox brushes extend up to it will divide it into smaller brushes and prevent it from crashing.
cp_bruhstbowl's skybox ceiling brush is 13312x13696 units, and it compiles just fine.

Because nearly every area is separated by these skybox brushes, even though cp_bruhstbowl has a fairly large amount of visleaves and a fairly small amount of func_detail geometry, it compiles with speed and performs well in-game.
Sometimes I look back at my artpassed maps and react with shock when I see the amount of things I've forgotten to turn into func_detail, and yet the map compiles pretty damn quickly...

Also, in case you're wondering about how I can have skybox brushes on top of the roofs without causing visual errors, that's one of the tricks I've picked up over the years:

Since the actual top part of the roofs are func_illusionary, they actually won't be cut by the skybox brush, and also won't be solid, so I don't even need to clip these roofs - the skybox brush does it for me.

2) 2020s redpills​

One thing that people were very interested in sharing around when I was getting started in TF2 mapping was that performance (in TF2, at least) is no longer based as much on raw polycount (as it was in 2007) as it is on draw calls - or, the number of models and textures your GPU is being told to draw.
This has a few significant implications:

• The downside of using Power 3 and Power 4 displacements is now negligible
• Spamming the same prop multiple times across a scene is now damning for performance, because TF2 doesn't have instanced rendering so you incur a new draw call for both the model and UV for every single usage of the prop
• Brushes are combined by VBSP (to an extent) into one large "model" per scene, so they don't suffer as badly from this
• Combining world brushes and func_detail into the same model is difficult, so scenes with mixed world brushes and func_detail are likely to perform worse

And, most important of all:

• Your map's draw calls are often DWARFED by the amount of draw calls it takes to render a playermodel - one for the actual model, one for its UV, another two for each of the cosmetics the player is wearing, another two for the weapon the player is holding, a variable number for Unusual effects

This means that even if your map is horribly optimised and you're rendering the entire map at once, the reason why it's performing badly isn't the actual detail of your scene - it's the fact that each player is rendering all other player's models at once.
If you've ever been on the playtesting servers and wondered why your framerate is so bad on a greybox - that's why.

The significance of draw calls over polycount also means that the effectiveness of various optimisation techniques have changed:

• Prop fade distances and func_lod were once an invaluable tool to reduce polycount in detailed scenes, but are now almost worthless
• func_areaportal and func_occluder were once expensive operations on the CPU and were advised to be used only rarely, but now are absolutely essential because their performance hit is much less significant, and they directly target derendering of props, and as we know props include playermodels and constitute the most significant performance hit in TF2

So, since func_occluder is so important now, let's teach you how it works, since many people have no clue.

This is a func_occluder on cp_bruhstbowl.
To create a func_occluder, start with a nodraw brush. Use CTRL-T to turn it into a func_occluder, then texture the faces you want to occlude stuff with the "occluder" texture.
Then, shove it inside the wall you want it to occlude.
But what does "occlude" mean exactly?
Well, to put it simply, an occluder surface will hide any props which are covered by it on your screen.
Here are (most of) the props that the occluder will be hiding from this angle:

As you can see, this one tiny brush is doing a pretty fucking stellar job of removing details we shouldn't be able to see from our scene.
ALL of these props would have been rendered if we didn't have an occluder here, because all the visleaves they're contained in are visible from the current visleaf, and visleaves tend to be so generous that even if you go crazy with hint brushes it won't help.

It's also worth noting that ALL faces of a func_occluder that have the occluder texture will act as an occluding face, so this occluder can trivially be made two-way:

There are, of course, a couple of caveats:

• A prop will only be occluded if its ENTIRE bounding box is within the func_occluder in screen space
• func_occluder tends to be bad at hiding brush models (world geometry, func_detail, displacements)
• If your func_occluder isn't entirely contained within a wall, then stuff seen through it will still be invisible, causing visual errors

But as long as you're aware of these, func_occluder is pretty awesome and easy to use, and I highly recommend using it wherever it will hide a large number of props.

In this same area, there's another pretty cool optimisation technique:

This door is a shortcut for RED to get to 1-A that closes once 1-A is captured.
Since this door closes and STAYS closed, we can use a clever trick called "closed areaportals" to prevent ANYTHING from rendering through it after A is capped. And, as it happens, areaportals natively support being linked to a door's open/closed state, so they'll render what's visible through the door when the door is open, and render nothing beyond the door when it's closed.

This, combined with the other areaportals in this brick building's exits, make it nigh-on impossible for anything inside the building - including players - to be rendered unless you're basically looking through one of its doorways.

3) Remember, no 3dsky​

Here's the vista from BLU's spawn for the second stage of cp_bruhstbowl, without the 3d skybox enabled:

Now here it is, with the 3d skybox enabled:

Yep, you're seeing that right. All it added was a few extra brushes, and a few tree props and a minehead.
(Admittedly, this vista is a little artistically uninspired and I may expand it one day.)

What I imagine you'd do normally is have skybox brushes just beyond the concrete walls and fences that're at the edge of the arena below. But I didn't want to do that, for several reasons:

• Whatever's in the 3D skybox will ALWAYS render, even if you're in a different part of the map and shouldn't be able to see it - thus hurting performance unnecessarily
• Since the 3D skybox is scaled up 16x, any brushes you put in there are essentially 16 times the luxel size they should be, and the lighting on them looks horrible
  • This also prevents you from using most props in the 3D skybox
• I wanted to have (limited) visibility between the first and second stages, similar to cp_dustbowl

Admittedly, you can solve the 2nd problem by simply setting the 3D skybox scale to 1, which I did in my later map koth_slums.

One thing the 3D skybox has traditionally been held to be useful for is creating landmarks which are visible from multiple areas of the map. But you really don't need the 3D skybox to do that, and I'm about to show you why.

See that building over there?

Well, you'd never know it in-game, but that building is actually halfway inside a skybox brush!

Normally, this would cut the building in half and leave it looking rather grotesque.
But this entire building is a func_brush, so it doesn't actually get cut in half by the skybox brush.
func_brush has a special quirk, which is that the entire func_brush will render if any part of its bounding box is visible.
So, this building is actually fully visible from both sides of this skybox brush.
If I had left it as normal world geometry and put it on only one side, the optimisation would have been so badass that it would have made it invisible from the other.

Props and displacements also share this bounding-box quirk, in case you're wondering.
So you can do the exact same thing with them.

However, this building actually has a few glaring flaws that led me to choose a different method of doing this in future:

So, in future, I started to use what I call "visibility crumbs", which are little nodraw cubes that are part of the func_brush but hidden inside solid geometry, and they only exist to extend its bounding box:

Since the water tower doesn't intersect the skybox brushes at all, any shadows on its surface are left perfectly intact - although nothing on the other side of the skybox brushes can cast a shadow onto it, which is why all the geometry on the other side is so low to the ground.

All these techniques ultimately allow me near-perfect control of what is visible in my map from where, such that the 3D skybox is virtually never needed - and good riddance, if you ask me!
Something I also forget to mention is that players can turn the 3dsky entirely OFF if they so choose, and many FPS configs DO.

Some of you may be thinking "but if I don't have a huge, expansive 3d skybox, players are gonna see visual errors when they rocket jump!"
This is true. Now, try running around in singleplayer on every single official TF2 map, and find me a single one where you can rocket jump without seeing some kind of visual error.
Spoiler alert - you're going to be searching for a while...

Another fun technique you can do to preserve lighting quality on a building that's visible through skybox brushes is to make use of triangular geometry:

4) 2020s optimisation is 2000s optimisation​

Let's look at a couple of community maps from 2009:

ctf_vector, by Icarus

pl_halfacre, by YM

Wow, both of these 2009 maps are even more visually gorgeous than my cp_bruhstbowl which came out in 2022, and they run well too!
What gives?

Well, as it happens, people were pretty clever in 2009, and knew all the optimisation techniques they could use to make a beautiful map run fast.
In fact they had to, since as I said earlier, raw polycount mattered more in 2009, so having highly detailed scenes necessitated not only heavy partitioning of areas, but also clever usage of optimisation tools within areas such as areaportals. I'm pretty sure func_occluder wasn't even in TF2 in 2009, as well...
And all they really had to do was follow Valve's example, since all six release-day TF2 maps are not only highly detailed, but also really heavily optimised to meet the computational restraints of 2007.

And this is the thing that aggravates me about present-day maps.
So many times, you see a mapper entirely neglect optimisation and just assume their map will naturally run well because "it's not 2009 anymore".
Then, they end up creating a scenario where almost their entire map ends up being rendered at once, including all playermodels on the server, and the map not only looks worse than a map from 2009 but also runs worse.

"It's not 2009 anymore" shouldn't be an excuse to NOT skybox off your roofs - it should be an excuse to add more detail AFTER you skybox off your roofs.
In that sense - and combined with intelligent use of lighting - you can achieve MUCH more detailed maps than the mappers of 2009 could have dreamed of, and with vastly better performance as the cherry on top.

One of the first things I tried to do after realising all this was to incorporate real-time reflective water into a map and have it perform at a decent framerate - that was pl_boatload.
Now, it's not like you don't get the occasional frame drop (below 120 fps) on pl_boatload if you have real-time reflections turned on - but as far as I know, it's the most performant TF2 map with real-time reflections out there, even beating several greybox maps and maps from 2009 - and this is without sacrificing any bit of my usual level of detail, in a gigantic 4-point single-stage Payload map.
That should speak volumes for the powers of hardcore optimisation, and the extra things you can add to a map when you're optimising it like you want it to run on 2009 computers.

 

[Brokkhouse]

I will refer to this guide frequently. Thank you so much, Tiftid.

 

[Plainz]

pretty good guide

 

[Yrr]

A lot of this is good, but the entire opening section about hinting is extremely misguided.

 

[Tiftid]

A lot of this is good, but the entire opening section about hinting is extremely misguided.

This is a fair point. Overhinting cannot hurt you in any way other than wasting your time.
Adding hint brushes to your map will pretty much always make performance better by some amount, and I'm not sure if I communicated clearly enough that my point is to not bother when that amount is tiny.

(I also describe later in the guide how if your skyboxing is good enough, the amount of visleaves on your map effectively doesn't matter, making hint brushes have minimal effect on compile time.)

It's a similar thing with prop fades. Making your props fade out at a distance will still benefit your performance even in 2024 - it's just not by a very large amount.

 

[MayaMogus]

The func_brush trick is genius, will finally allow me to be more open to skyboxing off more areas.

 

[iiboharz]

I think you should remove the section about hint brushes, it's not good and is pretty misguided as Yrrzy already mentioned. I also think the section about the 3D skybox could use some revision. This is otherwise a pretty solid guide.

 

[Brokkhouse]

The func_brush trick is genius, will finally allow me to be more open to skyboxing off more areas.

The badwater tower that separates B and Last may blow your mind

 

[Brokkhouse]

I think you should remove the section about hint brushes, it's not good and is pretty misguided as Yrrzy already mentioned. I also think the section about the 3D skybox could use some revision. This is otherwise a pretty solid guide.

I hang around in #small-questions-megathread a lot and a lot of questions are about optimization. Hint brushes are the go-to "solution" in what I call "Cargo Cult Optimization" - the use of optimization tools and entities not for their actual purpose, but because the mapper thinks just adding them to the map will magically make it run better. In that sense I really understand what Tiftid is going for here.

@Tiftid , don't remove the Hint section, but maybe reword it a bit. Feel free to use Cargo Cult Optimization as a term.

 

[Yrr]

Misusing a tool can be worse than not using it, but don't throw it out just because it might be used wrong.

 

[dabmasterars]

Good guide, but here are some things that I find partially wrong:

"Prop fade distances and func_lod were once an invaluable tool to reduce polycount in detailed scenes, but are now almost worthless"

You just said that saving draw calls is important. Less props to render - less draw calls for textures. Unless I got something wrong, this is a contradiction.

"Whatever's in the 3D skybox will ALWAYS render, even if you're in a different part of the map and shouldn't be able to see it - thus hurting performance unnecessarily"

1. When you are inside of a building or other part of the map where the skybox can't be seen, it unloads completely. (for example, on cp_well last)
2. Brushes inside of the 3d skybox are extremely cheap, since they don't care for visleaves. You can put a bunch of simple brush buildings and the performance impact will be nonexistent. look at cp_edge_v2
3. As for props rendering, you can put a prop fade on them, so they will not render in another side of the map. (prop fade distance should be 16x times lower that they usually are) It seems like prop fade IS important, after all.

If I got something wrong, please elaborate why.

 

[Tiftid]

Good guide, but here are some things that I find partially wrong:

"Prop fade distances and func_lod were once an invaluable tool to reduce polycount in detailed scenes, but are now almost worthless"

You just said that saving draw calls is important. Less props to render - less draw calls for textures. Unless I got something wrong, this is a contradiction.

"Whatever's in the 3D skybox will ALWAYS render, even if you're in a different part of the map and shouldn't be able to see it - thus hurting performance unnecessarily"

1. When you are inside of a building or other part of the map where the skybox can't be seen, it unloads completely. (for example, on cp_well last)
2. Brushes inside of the 3d skybox are extremely cheap, since they don't care for visleaves. You can put a bunch of simple brush buildings and the performance impact will be nonexistent. look at cp_edge_v2
3. As for props rendering, you can put a prop fade on them, so they will not render in another side of the map. (prop fade distance should be 16x times lower that they usually are) It seems like prop fade IS important, after all.

If I got something wrong, please elaborate why.

Hi,
It is true that fading out props will reduce your number of draw calls in a scene.
The reason why I did not mention this is because the semitransparency effect that happens between a prop's min and max fade distance is purported to be expensive, and generally you'll save a lot more props from rendering by using proper skyboxing and occludering than by labouring for hours setting fade distances.
You can, of course, do both, and should - if you're looking to squeeze every last frame out of your map.

A nice trick to prevent the semitransparency effect from happening is to leave the min fade dist at -1, so the prop fades out instantly.
This is pretty jarring, so I like to be overgenerous with my fade distances and only fade out my props when they DEFINITELY can't be seen.
However, this only leads me back to the pointlessness of prop fades - at the distances where my props can't be seen fading out, they'll usually also be in a place where some areaportal or occluder, or just good visleaf optimisation would be derendering them anyway.
So I either have to have visible and jarring prop fades, or no prop fades at all, and these days I tend to lean towards the latter lol.

I'd be curious to know how the game checks whether or not "the 3d skybox can be seen"; I guess it'd be "is there a skybox brush in the current PVS?".
I don't really get what you mean by "brushes inside of the 3d skybox are extremely cheap, since they don't care for visleaves." Obviously brushes are cheap, because VBSP combines them into a single model wherever possible.
I imagine you mean that since there are no visleaves in the 3d skybox, it's easy for VBSP to combine all the geometry therein into one large model, since we don't have to have different parts pop in and out depending on the number of visleaves.
This is fairly valid, but for the same reasons we can't just combine the entire map into one model and render it all at once - even in 2024 - this isn't always gonna be a great solution - for instance, when you're using materials that have different shaders, and therefore prevent VBSP from combining them into the same model.
Still, the benefit of the widespread combination is... probably noticeable?

What I said was that brushes in the 3d sky are more expensive than brushes in the main map, because they don't care for visleaves and so can't be optimised like main-map geometry.
Although, if prop fades work in the 3d skybox, func_lod probably does too, and if used in a map that would be 2009-pilled and I would be pleased.

There are many reasons why I don't use the 3d skybox much at all. I spoke on a few of them in the guide.
But, in truth, I never believed that the 3d skybox was THAT damning for optimisation - it can definitely harm your map's performance, but not nearly as much as some of the other mistakes you can make, such as not skyboxing off your map properly.
I oughta have been more clear on that.

 

[Another Bad Pun]

The advantage of 3D Sky geometry is that a player who doesn't want to render it can simply turn it off. If you do actually want vvis to cull it for some reason, you can use the compile parameter -forceskyvis.