DISCLAIMER: This article was originally written by MangyCarface for nodraw.net in 2009, and was rehosted here in 2012 due to malware infecting nodraw.net. Though nodraw.net has been safe to visit for many years now, this guide remains here as a mirror due to the value of the information. You can view the original article here.
__________________________________________________________________________
Optimization is one of the least understood parts of mapping in source; there is a lot of hearsay on the matter and generally the topic can appear overwhelming to new mappers. Today we’re going to discuss how optimization in source works, not from a theoretical approach, but from an example-by-example method that will hopefully bring us around to understanding the underlying theory. When implemented correctly, good optimization can be more beautiful than the aesthetics of a map itself.
This tutorial assumes you have an intermediate understanding of using hammer and the terminology used for its processes.
Part I: Visibility in Source
For this exercise we will be considering the source game TF2, though these techniques will apply to source in general. Our players will consist of a brick wall, an engineer representing the player, and a sentry representing the high-polygon props that end up creating framerate loss on our players’ end. Our goal today will be to instruct the engine not to draw the sentry when it is indeed hidden by the brick wall.
We would assume that, when looking at the wall, the engineer’s GPU would not be rendering the sentry. In source, not necessarily so! On compile, when vbsp is run, the engine makes imaginary cuts out of any open space to determine what players should render. Confused? Well, let’s look at the scene from above.
All entities are ignored in this “cutting”, which we call a “binary space partition.” Sound familiar? The initials, bsp, are found in our maps’ file type. Binary space partition sounds like a complicated phrase, but is easily illustrated by this example taken from Wikipedia:
The BSP process cuts the interior of your map into pieces, one by one, until each separate piece is no longer concave. Then, the visibility calculator (vvis.exe) imagines itself as each of these created pieces, and attempts to draw lines outwards to every other piece. If, from any position in one piece it has a line of sight to another, it considers that piece visible. Let’s look at it in context. Remember that world brushes, here drawn in red, make our square concave, so the engine will cut around it, in green:
Now let’s attempt to draw lines from the engineer’s partition to the sentry’s partition.
As you can see, even from the deepest angle, we can’t draw a line from the engy’s space to the sentry’s. Therefore, the engy won’t render the sentry! Hoorah!
But what if the engine had instead, arbitrarily, partitioned the scene starting across the wall‘s longer sides, like this?
As you can see, the created partitions are all convex and therefore valid, but look at this:
We can easily draw a line from the engy’s partition to the sentry’s, and so he would render it while looking at the wall! This is clearly incorrect, so how do we tell the engine to partition it like our first example, rather than this inferior second one? The answer lies within the tool texture, Hint. We use hint brushes to tell the BSP system- or rather, to hint to it- where it should perform its partitions first. A hint brush is constructed by texturing a world brush entirely with skip, and then applying the hint tool material along the face where we want a cut made. Let’s quickly add hints to our example so that it partitions correctly. We want to place them so that those cuts along the wall’s skinny edges are made first.
This is now correctly hinted, and presuming our scene had four walls and a ceiling in addition to the brick wall, would be cut as in our first partition example. So what can we take from this?
Part II: Optimization Structures
Hints
We will now go into a discussion of practical uses of hints. From here on out, we will be using cordons, marked in red, to represent the walls and ceilings surrounding our scene; red lines will represent world brushes, and purple represent the hint faces; yellow lines will denote possible lines of sight out of any one partition.
Our scene was actually one of the simpler uses of the most basic structure of optimization- the T structure. The T structure is formed by a wall perpendicular to another wall, along whose front a hint brush is placed. In hammer, the structure would look like so:
This is the simplest of optimization situations, and one that can be combined into many other derivative forms. The first of these derivatives is the H structure, which we’ve studied: A solitary wall bracketed by two hint brushes.
In this situation we’ve replaced one wall with a hint brush, so that players on each side of the hints can see down the hall, but those within the width of the wall cannot see the sentry.
We can combine this with a few T-like walls to create something more feasible:
We now have 4 Ts jutting out from our cordon walls to touch the hint brushes. This means that players in the halls can see both areas, but any players in the partitions behind any of the 3 walls on the engineer’s side cannot see into the sentry’s partition. We’ll call this shape the ]-[ configuration.
The next shape, the S shape, is created out of 2 individual Ts. It is frequently used within tunnels to prevent line of sight issues and help optimize in this fashion:
We’ve added another pair of engie and sentry to demonstrate the dual quality of this simple arrangement. The bottom engie cannot see either sentry, and the top engie can see 1 but not both.
We can add in a third T to create a more solid optimization corridor, in the W shape:
This should look familiar, as the W is basically half of our ]-[ shape, but designed to work with only a single path instead of two paths of travel.
If we rotate the W to be on a diagonal and then snap it to the grid, we come upon another oft-used optimization shape, one that is used in many corners. We’ll call it the L shape. First, we rotate our W:
Now we vertex manipulate it to come out with a more familiar pathway:
As you can see, we now have a right angle corridor, but even though our walls have changed shape the W’s optimization is still at work. Only players inside of the green triangle can see both areas. We can use the L shape in a variety of places. An easy example is the right-most corridor on 2fort as you enter the enemy base from the outside second-floor; it leads to their spawn room area.
It is important to note in all of these cases that we have been assuming a solid world brush ceiling. This will usually not be the case in your maps! But luckily, the T structure can be used in horizontal situations as well, and combined with all of the other shapes to form their ceiling. Only players below the horizontal hint will benefit from the vertical optimization shapes, so make sure that if players will be above the horizontal hint, it will only be shortly. A simple use:
Our T is still intact, perpendicular to the floor; in fact, in this position you can see where it derives its name. Let’s see it in combination with the W:
So while there is no ceiling abutting our shorter walls, the T function of the horizontal hint allows the W to still work correctly. It is important to note that without a ceiling or horizontal hint, there is no telling how vbsp and vvis will interpret your optimization structure! If any of the partitions extends above the walls involved with the particular optimization structure, VVIS will be able to see down from the top of the partition into other partitions, and players in that first partition will therefore render objects otherwise hidden by walls.
Part III: Examples of Hint Structures in Use
We have now covered all of the basic optimization shapes that involve hints. Experiment with combining Ts into other new structures to complement your map design. OPTIMIZATION MUST BE CONSIDERED WHEN CONSTRUCTING A LAYOUT. These structures are simple but need to be in mind when laying out a map! Remember that any of these structures can be scaled to any size, and we will demonstrate macro uses of these micro shapes in this next section: real examples. I’ll be using mostly my own work to point it out; you may download them if you wish to see these in game. Our actors will be represented by two blue circles; in each scenario the circles will not render one another.
Basic T:
Here’s an example from Yukon’s CP1 area of a Basic T in action. This demonstrates how large a scale you can use it on, and how you can vary the thickness of the T’s trunk:
Basic H:
Again from Yukon, this is the upper transition from CP2 to CP3:
You can see how large you can expand these structures to be without interrupting function.
S Shape: Atrophy, near B and A, we have an example of a larger S-shape. Blue cannot see one another, and red cannot see one another; additionally, the top left blue cannot see the bottom right red:
W Shape: The W is one of my favorite; it can be implemented on an enormous scale like here in cp_atrophy, between B and C:
L Shape: This is easy to implement in simple fashion, but very cool to use in a different manner like here in Yukon, in the alternate path between CP3 and CP2 (cliffs removed for visibility):
Here’s another even larger example in Atrophy between A and C:
Vertical T: These are everywhere, but a simple example is in ctf_aurora, where a horizontal hint covers the entirety of the map and guarantees that players in the backyards cannot see those in the middle:
There are unlimited examples of such structures being twisted and used in every imaginable fashion, so keep an eye out! I couldn’t possibly cover each instance.
Part IV: Other Optimization Brushes, a Primer
Area Portals
Now we will cover, in short, areaportals in their door function. Area portals are brush entities, completely covered in the areaportal material and tied to a func_areaportal. Area portals act like world brushes when closed, and like open space (with some restrictions) when open. We use area portals within doors to prevent players from drawing anything inside the doored area. However, the area behind the area portal must be its own ‘area’ in the sense that, other than through another area portal, no VVIS lines (our yellow arrows) can be drawn into a partition on the outside of the area. A simple way to test this is to start from any visible face of your area portals and try to navigate through empty space to another face of the same area portal. If you’re able to do so without entering another area portal, you need to seal the area better! But if you cannot, then your area is well-sealed and ready for compile. Area portals throw leaks if not sealed correctly, and can be a pain to sort out, but that’s too indepth for our brief overview here. Let’s look at a simple case of using an area portal:
Here we have our original scenario, but with the wall extended to fit against our cordon bounds and a func_door added right between our two actors. Remember that a door is an entity and therefore is excluded from the vbsp process; it cannot by itself block visibility. This is where the area portal comes in: we place it within the door such that it is entirely enclosed and touching all 4 faces (3 brick, 1 floor) around the door. We then set its properties so:
If your door starts closed, as would ours, we also start the area portal as closed. We then set its linked door to be the name of our door. Now, when the door’s position is toggled, the area portal will toggle between open and close: once the door is fully closed, our area portal will shut and act like a world brush and the engineer will stop rendering the sentry. These are best used with spawn doors and other doors that will only be open momentarily; this gives you a higher budget (budget in optimization terms is the amount of detail you can use without impacting framerates negatively) to work with, without using one of our hint structures (which tend to complicate areas) instead.
Occluders
There is one final brush entity we need to know when optimizing (in TF2; other games use area portal windows, which we can explain at a different time.) This entity is called a func_occluder. It acts as a curtain between you and props on its opposite side; each side of an occluder entity textured with the occluder tool material will prevent you from rendering any objects completely covered by that face. Let’s take a situation where we have a func_detail wall instead of a world brush wall. It’s ignored in visibility calculations, but we still want our player not to render the sentry. Here is our scenario:
In this case we don’t want players at the sentry’s position to render the engy either, so we’ll make an occluder with two faces textured with Occluder. Like a hint brush, its other faces will be skip. We create this brush either within the wall or exactly as big as the wall:
We’ve toggled off details in our visgroups so that we can see the Occluder here. Now, neither the sentry nor the engy, were they both players, would be able to render one another. Occluders sound great, but they’re not at always a good idea! It costs the client to calculate which props should be ignored, so be sure that each occluder prevents a large number of high poly props from being drawn to balance out the initial cost of calculation. Look to Gravelpit’s vmf for correct usage.
Part V: Other Ways to Optimize
This concludes our overview of brush-based optimization, but there are a few other ways to optimize we need to keep in mind. These are kind of like cheating, and you’ll only squeeze scant frames out of each, so be sure to use hints and all before you go through and do these:
Part VI: Closing, Extra Links, and Glossary
We have looked at a variety of ways to optimize your maps, but there are more details to be learned! Now that you have a basic knowledge, search the VDC and elsewhere for specifics on theory and indepth explanations of any of the aforementioned knowledge areas. I hope you are able to come out with the next great work, and one that will perform well on all machines! Look for combinations of what we’ve examined, and be clever; the more you press yourself to be creative in optimizing, the less of a chore it’ll feel- imagine it a challenge.
LINKS:
__________________________________________________________________________
Optimization is one of the least understood parts of mapping in source; there is a lot of hearsay on the matter and generally the topic can appear overwhelming to new mappers. Today we’re going to discuss how optimization in source works, not from a theoretical approach, but from an example-by-example method that will hopefully bring us around to understanding the underlying theory. When implemented correctly, good optimization can be more beautiful than the aesthetics of a map itself.
This tutorial assumes you have an intermediate understanding of using hammer and the terminology used for its processes.
Part I: Visibility in Source
For this exercise we will be considering the source game TF2, though these techniques will apply to source in general. Our players will consist of a brick wall, an engineer representing the player, and a sentry representing the high-polygon props that end up creating framerate loss on our players’ end. Our goal today will be to instruct the engine not to draw the sentry when it is indeed hidden by the brick wall.
We would assume that, when looking at the wall, the engineer’s GPU would not be rendering the sentry. In source, not necessarily so! On compile, when vbsp is run, the engine makes imaginary cuts out of any open space to determine what players should render. Confused? Well, let’s look at the scene from above.
All entities are ignored in this “cutting”, which we call a “binary space partition.” Sound familiar? The initials, bsp, are found in our maps’ file type. Binary space partition sounds like a complicated phrase, but is easily illustrated by this example taken from Wikipedia:
The BSP process cuts the interior of your map into pieces, one by one, until each separate piece is no longer concave. Then, the visibility calculator (vvis.exe) imagines itself as each of these created pieces, and attempts to draw lines outwards to every other piece. If, from any position in one piece it has a line of sight to another, it considers that piece visible. Let’s look at it in context. Remember that world brushes, here drawn in red, make our square concave, so the engine will cut around it, in green:
Now let’s attempt to draw lines from the engineer’s partition to the sentry’s partition.
As you can see, even from the deepest angle, we can’t draw a line from the engy’s space to the sentry’s. Therefore, the engy won’t render the sentry! Hoorah!
But what if the engine had instead, arbitrarily, partitioned the scene starting across the wall‘s longer sides, like this?
As you can see, the created partitions are all convex and therefore valid, but look at this:
We can easily draw a line from the engy’s partition to the sentry’s, and so he would render it while looking at the wall! This is clearly incorrect, so how do we tell the engine to partition it like our first example, rather than this inferior second one? The answer lies within the tool texture, Hint. We use hint brushes to tell the BSP system- or rather, to hint to it- where it should perform its partitions first. A hint brush is constructed by texturing a world brush entirely with skip, and then applying the hint tool material along the face where we want a cut made. Let’s quickly add hints to our example so that it partitions correctly. We want to place them so that those cuts along the wall’s skinny edges are made first.
This is now correctly hinted, and presuming our scene had four walls and a ceiling in addition to the brick wall, would be cut as in our first partition example. So what can we take from this?
Part II: Optimization Structures
Hints
We will now go into a discussion of practical uses of hints. From here on out, we will be using cordons, marked in red, to represent the walls and ceilings surrounding our scene; red lines will represent world brushes, and purple represent the hint faces; yellow lines will denote possible lines of sight out of any one partition.
Our scene was actually one of the simpler uses of the most basic structure of optimization- the T structure. The T structure is formed by a wall perpendicular to another wall, along whose front a hint brush is placed. In hammer, the structure would look like so:
This is the simplest of optimization situations, and one that can be combined into many other derivative forms. The first of these derivatives is the H structure, which we’ve studied: A solitary wall bracketed by two hint brushes.
In this situation we’ve replaced one wall with a hint brush, so that players on each side of the hints can see down the hall, but those within the width of the wall cannot see the sentry.
We can combine this with a few T-like walls to create something more feasible:
We now have 4 Ts jutting out from our cordon walls to touch the hint brushes. This means that players in the halls can see both areas, but any players in the partitions behind any of the 3 walls on the engineer’s side cannot see into the sentry’s partition. We’ll call this shape the ]-[ configuration.
The next shape, the S shape, is created out of 2 individual Ts. It is frequently used within tunnels to prevent line of sight issues and help optimize in this fashion:
We’ve added another pair of engie and sentry to demonstrate the dual quality of this simple arrangement. The bottom engie cannot see either sentry, and the top engie can see 1 but not both.
We can add in a third T to create a more solid optimization corridor, in the W shape:
This should look familiar, as the W is basically half of our ]-[ shape, but designed to work with only a single path instead of two paths of travel.
If we rotate the W to be on a diagonal and then snap it to the grid, we come upon another oft-used optimization shape, one that is used in many corners. We’ll call it the L shape. First, we rotate our W:
Now we vertex manipulate it to come out with a more familiar pathway:
As you can see, we now have a right angle corridor, but even though our walls have changed shape the W’s optimization is still at work. Only players inside of the green triangle can see both areas. We can use the L shape in a variety of places. An easy example is the right-most corridor on 2fort as you enter the enemy base from the outside second-floor; it leads to their spawn room area.
It is important to note in all of these cases that we have been assuming a solid world brush ceiling. This will usually not be the case in your maps! But luckily, the T structure can be used in horizontal situations as well, and combined with all of the other shapes to form their ceiling. Only players below the horizontal hint will benefit from the vertical optimization shapes, so make sure that if players will be above the horizontal hint, it will only be shortly. A simple use:
Our T is still intact, perpendicular to the floor; in fact, in this position you can see where it derives its name. Let’s see it in combination with the W:
So while there is no ceiling abutting our shorter walls, the T function of the horizontal hint allows the W to still work correctly. It is important to note that without a ceiling or horizontal hint, there is no telling how vbsp and vvis will interpret your optimization structure! If any of the partitions extends above the walls involved with the particular optimization structure, VVIS will be able to see down from the top of the partition into other partitions, and players in that first partition will therefore render objects otherwise hidden by walls.
Part III: Examples of Hint Structures in Use
We have now covered all of the basic optimization shapes that involve hints. Experiment with combining Ts into other new structures to complement your map design. OPTIMIZATION MUST BE CONSIDERED WHEN CONSTRUCTING A LAYOUT. These structures are simple but need to be in mind when laying out a map! Remember that any of these structures can be scaled to any size, and we will demonstrate macro uses of these micro shapes in this next section: real examples. I’ll be using mostly my own work to point it out; you may download them if you wish to see these in game. Our actors will be represented by two blue circles; in each scenario the circles will not render one another.
Basic T:
Here’s an example from Yukon’s CP1 area of a Basic T in action. This demonstrates how large a scale you can use it on, and how you can vary the thickness of the T’s trunk:
Basic H:
Again from Yukon, this is the upper transition from CP2 to CP3:
You can see how large you can expand these structures to be without interrupting function.
S Shape: Atrophy, near B and A, we have an example of a larger S-shape. Blue cannot see one another, and red cannot see one another; additionally, the top left blue cannot see the bottom right red:
W Shape: The W is one of my favorite; it can be implemented on an enormous scale like here in cp_atrophy, between B and C:
L Shape: This is easy to implement in simple fashion, but very cool to use in a different manner like here in Yukon, in the alternate path between CP3 and CP2 (cliffs removed for visibility):
Here’s another even larger example in Atrophy between A and C:
Vertical T: These are everywhere, but a simple example is in ctf_aurora, where a horizontal hint covers the entirety of the map and guarantees that players in the backyards cannot see those in the middle:
There are unlimited examples of such structures being twisted and used in every imaginable fashion, so keep an eye out! I couldn’t possibly cover each instance.
Part IV: Other Optimization Brushes, a Primer
Area Portals
Now we will cover, in short, areaportals in their door function. Area portals are brush entities, completely covered in the areaportal material and tied to a func_areaportal. Area portals act like world brushes when closed, and like open space (with some restrictions) when open. We use area portals within doors to prevent players from drawing anything inside the doored area. However, the area behind the area portal must be its own ‘area’ in the sense that, other than through another area portal, no VVIS lines (our yellow arrows) can be drawn into a partition on the outside of the area. A simple way to test this is to start from any visible face of your area portals and try to navigate through empty space to another face of the same area portal. If you’re able to do so without entering another area portal, you need to seal the area better! But if you cannot, then your area is well-sealed and ready for compile. Area portals throw leaks if not sealed correctly, and can be a pain to sort out, but that’s too indepth for our brief overview here. Let’s look at a simple case of using an area portal:
Here we have our original scenario, but with the wall extended to fit against our cordon bounds and a func_door added right between our two actors. Remember that a door is an entity and therefore is excluded from the vbsp process; it cannot by itself block visibility. This is where the area portal comes in: we place it within the door such that it is entirely enclosed and touching all 4 faces (3 brick, 1 floor) around the door. We then set its properties so:
If your door starts closed, as would ours, we also start the area portal as closed. We then set its linked door to be the name of our door. Now, when the door’s position is toggled, the area portal will toggle between open and close: once the door is fully closed, our area portal will shut and act like a world brush and the engineer will stop rendering the sentry. These are best used with spawn doors and other doors that will only be open momentarily; this gives you a higher budget (budget in optimization terms is the amount of detail you can use without impacting framerates negatively) to work with, without using one of our hint structures (which tend to complicate areas) instead.
Occluders
There is one final brush entity we need to know when optimizing (in TF2; other games use area portal windows, which we can explain at a different time.) This entity is called a func_occluder. It acts as a curtain between you and props on its opposite side; each side of an occluder entity textured with the occluder tool material will prevent you from rendering any objects completely covered by that face. Let’s take a situation where we have a func_detail wall instead of a world brush wall. It’s ignored in visibility calculations, but we still want our player not to render the sentry. Here is our scenario:
In this case we don’t want players at the sentry’s position to render the engy either, so we’ll make an occluder with two faces textured with Occluder. Like a hint brush, its other faces will be skip. We create this brush either within the wall or exactly as big as the wall:
We’ve toggled off details in our visgroups so that we can see the Occluder here. Now, neither the sentry nor the engy, were they both players, would be able to render one another. Occluders sound great, but they’re not at always a good idea! It costs the client to calculate which props should be ignored, so be sure that each occluder prevents a large number of high poly props from being drawn to balance out the initial cost of calculation. Look to Gravelpit’s vmf for correct usage.
Part V: Other Ways to Optimize
This concludes our overview of brush-based optimization, but there are a few other ways to optimize we need to keep in mind. These are kind of like cheating, and you’ll only squeeze scant frames out of each, so be sure to use hints and all before you go through and do these:
- Draw distances. You can assign fading properties to any props that will prevent them from being drawn at a certain distance away. This works best on small detail props, but beware that it costs the client to fade the prop, so they will only experience benefits when the prop is completely invisible.
- Func_lod. This is an entity that is exactly like a func_detail except that it has fade distances like props. Use these sparingly.
- Lightmap optimization. Look it up elsewhere. Doesn’t offer any huge gains, but shrinks file size.
- Func_detail. The less partitions created by vbsp, the shorter vvis takes and the less calculations your players have to perform within each partition. Use func_detail to reduce the complexity of your vbsp structure. To view the structure of your map, compile it and then go to Map->Load Portal File and choose the associated file. The blue lines are the cuts that have been made around each partition!
- Advanced area portal usage. Area portals have small performance benefits when placed in doorways correctly; again, find a different tutorial for this particular piece.
- Fog. At the limits of your map’s fog distance, decided by your env_fog_controller, nothing more is rendered; if you’re aggressive with this distance, you can slightly reduce framerate loss. Not advised.
- Water. You can use cheap water instead of expensive water (I presume you know what these are)
- Displacement power. If you can reduce the power of your displacements (working with power 2 is pro!) by even 1 degree, you will drastically reduce the energy needed to render that particular face. Remember that power 4 represents 8x the number of vertexes in power 2, 2x power 3. Even going from power 3 to 2 reduces the number of vertices considerably.
- Nodraw. The engine automatically deletes brush faces facing the void or completely covered by other brush faces, but there are often faces that do not satisfy these conditions that will never be seen by your players- usually the back faces of detail behind player clips. Applying the nodraw material to these faces will remove them from the cost of rendering.
Part VI: Closing, Extra Links, and Glossary
We have looked at a variety of ways to optimize your maps, but there are more details to be learned! Now that you have a basic knowledge, search the VDC and elsewhere for specifics on theory and indepth explanations of any of the aforementioned knowledge areas. I hope you are able to come out with the next great work, and one that will perform well on all machines! Look for combinations of what we’ve examined, and be clever; the more you press yourself to be creative in optimizing, the less of a chore it’ll feel- imagine it a challenge.
LINKS:
- VDC
- Great guide with more theory
- Maps used in discussion:
Last edited:
