Questions about optimization

[Mouse]

Hello I'm Mouse, I've been mapping for tf2 for over 8 years but have never
completed or released any maps. I'm very talented and I've got a question
for all you passionate and respectable mappers out there regarding optimization.

For the purpose of defining the conditions I will offer a scenario:
There is a map and it is fully completed, compiles and is playable without any errors.
This map contains thousands of brushes, which create even more visleafs.
Some of the brushes, the detail work, has bee made into func_detals thus generating no visleafs.

Now my question is this: Has it ever been attempted, and/or what would be the hypothetical outcome,
of turning every brush in the entire map into func_detail entities, enclosing the map in a skybox,
and manually creating hint/skip brushes to generate the cuts necessary to have visleafs?

Would this not reduce the amount of visleafs from thousands to merely tens of cuts?
Wouldn't tens of cuts be all that is necessary to limit what is rendered from any position?
Would this not reduce compile time significantly while increasing fps for players in game?

In this scenario the skybox could simply be a box surrounding the map to prevent leaks.
Hint/skip brushes would define the visleafs instead of doing it by visible world geometry.
I have had this question for a long time and would appreciate any well thought out answers.
Thank you in advance for your time/energy in answering these questions for me.

 

[ics]

If you func_detail your whole map, every visleaf will be drawn out, not to mention very much lowered framerate. Visleafs count is not the issue, its how many gets drawn out at the same time. This is why you use world geometry in addition with skybox texture to create smaller areas that you break apart from other such areas with areaportals.

Hint/skip brushes are ment to be used in addition to world geometry, to make vis cuts in areas that the engine cannot handle that well, such as horizontal brushes. If you get too many areas with automatic compile and it takes long time, you need to simplify world geometry and use more func_detail on areas where they are necessary.

 

[Mouse]

Ok I think I see what it is you're saying here.
Now what if instead of in my example how I said the map would be surrounded
by one large box keeping it from leaking, it was instead surrounded by many
boxes each forming "rooms" around all the other brushes, each defining it's
own area and coming together to seal off the map area, connecting each box
where the playable space moves through it in a corridor with areaportals.
I would then be able to tie every geometry in the map to a func_detail entity
except for those boxes and it would then perform the way I was intending
in my original idea? This way a map with thousands of brushes could
be compiled with only tens of visleafs?

 

[Yrr]

Would this not reduce the amount of visleafs from thousands to merely tens of cuts?
Wouldn't tens of cuts be all that is necessary to limit what is rendered from any position?
Would this not reduce compile time significantly while increasing fps for players in game?

no

I'm very talented

no

 

[LeSwordfish]

The point of Visleaves is to control which elements of the map are rendered and which aren't. Cutting down on visleaves means that more things the player can't actually see will be rendered needlessly, which is worse for optimisation than only showing what's actually needed.

Reducing compile time is mildly useful as a mapper, but means nothing to players. The more that the engine can work out once, in advance, when compiling, the less it needs to do dynamically, slowing down the game - that's why lighting, visflow, etc, is all done at compile rather than every frame during the game.

 

[Another Bad Pun]

Why do you want to cut down on the amount of visleafs vvis creates?

In cases where it'd create a huge mess, that's understandable. Those situations are what func_detail is for.

Beyond that, the number of visleafs created in your level will only effect your vvis compile time. If you create your map with optimization in mind by using clean brushwork, making sure there are no nasty lines of sight, and using consistent / larger grid sizes, it's possible to create a map that will run well for players and compile vvis in only a few seconds. I think that's the best way to create a well optimized map, if your layout allows it.

 

[tyler]

Wouldn't tens of cuts be all that is necessary to limit what is rendered from any position?

You are confusing cuts with culling. Cuts do not cull. They simply tell the engine where a specific cut can be continued, stopped, etc. In your hypothetical scenario, you would need to use a bunch of func_occluders to cull objects, which would absolutely destroy FPS. This old guide might be useful for explaining this idea.

Your idea of breaking a large "room" into smaller "rooms" is exactly how optimization actually works. You're just taking it way too far. Or, you can think of it as taking it not far enough. As the guide I linked showed, you can reduce a small "room" in a T or H shape into several smaller "rooms" (using your definition of a room).

You are essentially confusing visleaf complexity with optimization, which in TF2 is not really the most relevant factor. It's more important to tell the client what to render in highly specific ways. While reducing visleaf density can be important in some instances, it's pretty rare and only comes into play on highly detailed, highly complex maps. You actually want a lot of visleafs, so that portions of a level can be rendered or not rendered more easily. There is no sense rendering a huge area if the client can only see 25% of it.

To get an idea of how levels render in game, I suggest opening a map in TF2, enabling cheats, and playing with these commands:

mat_leafvis 1
r_drawstaticprops 2
r_drawportals 1
r_drawfuncdetail 0

Check the Valve Developer Wiki for more commands like this.

 

[Mouse]

definately getting current project
compile times down now, through
my testing process I put everything
in a large box so simply to seal it
so I can run it and verify it works.

well I saw somewhere something
about someone had laid they're
entire floor out as displacement,
and it got me thinkin, lookin at all
the lines and it was obvious. yeah
i just hadn't been taking enough of
my geometry from the rough up
stage to becoming func_detail or to
add displacements where they're
needed. my maps already are built
with these concepts in mind, i do
take it pretty far i guess. glad to
see this place leading to results!

 

[iiboharz]

Not gonna lie I'm having a very hard time reading any of your posts because they come across as a very disorganized stream-of-consciousness just being dumped into the thread.

I think what's important to note here is that while having long compile times can be frustrating, isn't necessarily a bad thing. While your methods suggested may possibly cut your compile times down to near zero, the final result will most definitely perform horrendously in-game. That said, cutting down your compile time is always desirable, just not when the method prioritizes short compiles over good map layout or in-game performance.

I don't think the proposed method here is really worth much thought. Just learn proper optimization theory and technique, and your compile times should be short, and your map should perform well in-game.

 

[Mouse]

well I didnt really ask for anyone
to judge me as a mapper or as
a person now did I lol. thanks for
all the information that I did get
about the questions that I did ask.

 

[Mouse]

Anyone is open to give me as detailed an interpretation on this information as they're
willing and able to: keep in mind these were both fast vvis compiles whatever that means.

Upward Decompiled: vvis stats.......................................My Secret Project Map: vvis stats
Total Clusters Visible: 1,519,944 ....................................Total Clusters Visible: 143,346
Average Visible Clusters: 816......................................Average Visible Clusters: 374
visdatasize: 774,995 compressed from: 893,280........visdatasize: 39,848 compressed from: 36,768
'optimized': 126,231 (8.31%).........................................'optimized': 1153 (0.80%)

Also note that my map is still in the testing phase which for me means it's still only a rough work,
comprised of about 70% displacements, 20% func_details, and 10% world geometry
and that this is all enclosed in one large map sealing skybox.
When I get the whole thing the way I want it, it will be fully enclosed in the normal tunneling way,
basically like a giant funky toroid, or C-shape, much like Upward which is why it's example A.

Cmon, serious feedback only. Do you think the skybox is what's causing optimization
percentage to be so low? I do.. and if it's the case that changing this will fix it IDC.
since it's already part of the plan to do so and would be silly if left undone.
This is not my complete secret project map, it is about 50% done block wise.
This means when I'm done, the percentage above will be more like 20% world geometry

Here are some ratios I calculated for reference in comparison:
Upward : Secret Project
10.6 : 1 Ratio of Total Visible Clusters
2.18 : 1 Ratio of Average Visible Clusters
My Map 0.09% As many clusters 'optimized' as Upward
Upward 10.33% As many clusters 'optimized' as My Map

Again for me this all seems perfectly normal, but as I learned the entire hammer program
myself and in my own way, I am sure that my style differs heavily from most.
Being here has definitely got me thinking in new and better ways, on top of what I know
and I do appreciate any support and feedback to help me streamline and fix my process
as well as to be open about my strategy and style such that anyone can learn from me.
At the moment my map is secretly about 75% the playable size as Upward.

 

[FloofCollie]

You can use the cordon tool instead of manually making a hollow skybox cube around your map - it can be flicked on and off with a simple click and easily resized like a brush so it's less hassle than the former method.

well I didnt really ask for anyone
to judge me as a mapper or as
a person now did I lol. thanks for
all the information that I did get
about the questions that I did ask.

I don't think iiboharz was being rude (at least not intentionally); it wasn't a personal attack or judgement, rather constructive criticism.

 

[Mouse]

Interesting.. I didn't know that about the cordon tool. Since I have never used it,
is this that red circle showing up on every map I've worked on in the editor?
is it actually capable of cordoning off an area defined by a skybox texture
that is a sphere? or does it somehow make a cube out of the circles I can see?
is it just that I have never set it manually so it's just a circle shape by default
and is actually not doing anything since I haven't set it up? definitely useful..

 

[FloofCollie]

No, that's Radius Culling and you should turn it off since everything outside the sphere is hidden and ignored in compiles, which can lead to leaks etc.

lt's meant to improve performance in Hammer but is usually an unnecessary annoyance when you forget about leaving it on. Click the button with an 'R' inside a red circle to turn it off.

The cordon tool looks like this and can be adjusted with this .

The bounds of the cordon box are treated like a skybox textured brush when compiling.

More info: https://developer.valvesoftware.com/wiki/Hammer_Cordon_Usage

Keep in mind that in Hammer, everything outside the cordon bounds is hidden, so if half your map disappears, that's why; you can just resize the box.

 

[Mouse]

Ah, yes I just realized this difference. Thanks for the clarification though!

 

[tyler]

"Optimization percentage" as you're interpreting it based on what you're reading in the compile log isn't what you think it is. If the compiler had a way to judge how well optimized your map is, it could just optimize it for you.

The compile log is not a grade you're receiving. It's primarily a tool for troubleshooting. Nobody ever looks at those numbers and thinks "I need to do ten percent better on visclusters" or whatever you're doing here. Use the in game tools during game play (real or simulated) to measure optimization.

 

[iiboharz]

In agreement with tyler, these "optimized" numbers you're mentioning have no bearing on anything relevant to map design. All the stuff displayed there in the compile log is in reference to the compile process itself, not the map.

If your map is really only 10% world geometry as you say, you are most certainly doing something wrong.

I reiterate that the concept and method proposed isn't worth much thought, as it goes back on over 20 years of level design in Quake engine derived games. If what you're trying to solve or think about here hasn't been completely thought about in the last 20 years I don't think we would be having this conversation.

 

[Mouse]

It most certainly does make a difference, as I have always done all my work on computers with really
slow CPU speeds and I have ever owned a computer that uses a GPU, saving on compile time and
also making a map with the most reduced lag possible has always been a primary goal of mine.
Seeing as how I am doing it this way, it is absolutely true that if my computer runs these maps well
that a powerhouse of a gaming rig that most people will be running it on will certainly handle it
even better. That being said I do not imagine to know what it would be like to have a GPU or to be
able to move things through a processor faster than 2 threads at 2.4Ghz, so honestly that kind of
computing and any strategies that come with it (or disappear because of it) really don't matter to me.

Now in my comparison between the map that I created and the map made by Valve, Upward - which
is easily the most popular tf2 map ever created, the difference in design should be so obvious that
I shouldn't even have to point it out. When making Upward, the devs at Valve made over one and a half
million visclusters, used 29/0 occluders, and wound up with 53,912 total triangle count. My map however
is being made with only a couple hundred thousand visclucters, using no occluders, and with a total triangle
count of 7,524 so far. Seeing as my map has an estimated 75% playable volume as Upward, and took me
9 minutes and 16 seconds less time to compile when doing these tests, with vvis on fast compile.. my map
is inarguably lower polycount, easier to work with, and does 100% function just as well at keeping areas
from being rendered as Upward. My count of 374 average visible clusters so far as compared to Valve's
average of 816 shows that my map does a better job at keeping the processing speeds up for players
while they're running around on the map in-game than Upwards, and I'm using an expensive box skybox.

I think the idea from Valve is that, with all the extra work they were willing to put into telling the engine even
more information about where to create visleafs instead of taking an approach similar to mine, has actually
caused them to lose sight of optimization teqhniques that when employed greatly reduce the amount of work
necessary to direct the engine. For instance I stated that 70% of my map project is displacements. They do not
generate visleafs but they do seal maps (proven to be false! my bad)
so they have been my go-to tool for bringing down the amount of clusters
outdoor and indoor. If my style works then it is worth a closer look at, regardless of how anyone feels about the
situation. No offense of course, but your comments iiboharz only demonstrate to me that you seem to have very
little understanding of the actual processes at work here, you are quick to jump to conclusions and that your opinion
that the business of mapping has only advanced in the last 20 years is detrimental to real progress to say the least.
However I have no problem with what you said, because it has inspired me to write out this lengthy explaination.

Cheers! I think that this thread has truly dually suited me.

 

[Da Spud Lord]

For instance I stated that 70% of my map project is displacements. They do not
generate visleafs but they do seal maps so they have been my go-to tool for bringing down the amount of clusters
outdoor and indoor.

(Emphasis mine)

This is patently false: Displacements do not seal maps at all. It is entirely possible for a map to leak through a displacement.

You say your map has significantly less triangles than Upward. The number of triangles in your map is not a sign of how well optimized your map is, but rather a sign of how much stuff is in your map. Although I can't be sure as I haven't seen your map, I would guess that your map contains significantly less detail than Upward. The improved performance you get on your map isn't due to better optimization, but rather due to simply having less stuff in your map that needs rendering. I would guess that if your map had detailing comparable to Upward, or if your optimization technique was applied directly to Upward without changing the visuals, we'd see a significant difference in results.

Also, and while I'm no technical expert, I don't think it would even be possible for TF2 to run on a computer without a GPU, and I'm pretty sure no PC made in the past 10 or more years doesn't have a GPU unless it was custom-built by you with the express purpose of not having a GPU. Even cheap laptops have an integrated GPU (a fairly weak GPU built into the motherboard).

 

[Mouse]

Well I do not have any sort of GPU capabilities listed in any form in my system specs, however my computer
does run tf2 incredibly well for what it is. I'm currently working on an HP Stream that I got at Walmart for $200.

And you are most definitely correct that at this point in the development of my map it is not yet fully detailed
and will require another pass over the whole thing to add these details in. however since they will all be made
into func_details this fact alone has no impact on framerate. It is true that I have a bit of a different style than
Valve when it comes to props and other details. Valve likes to load maps up with props as if it makes no difference
how many triangles are in each gamespace, while I on the other hand value the gameplay itself over the use of
expensive scenery aspects. I will actually use a good number of props to get the desired look by the end of it,
and thus all the effort I can put into simplifying and compartmentalizing each playable area of the map is going to
make a significant difference when rendering those surfaces.

Displacements may not seal maps, however using one to five blocks of world geometry, being
in position under and around such displacement areas, connecting to other world geometry or
skybox elements does seal such spaces and creates again far less visleaves per space,
cutting down the amount of calculations for both compile time and players in-game.
This is my strategy, excuse me. I did misspeak on that part. I cover the hundreds of displaced
blocks that make up the actual gamespace with behind the scenes world geometry in order to
keep the numbers as low as possible. I would stress that, for pretty much any space, only a few
to tens of visleafs are actually necessary to do the kind of occlusion we are talking about here.
Valve on the other hand with hundreds of thousands on one single stage of payload is crazy.
It is merely a testament to the creative process that a map which is essentially a toroid, or
large C shape as I stated above would need hundreds of thousands of visleaf clusters and
29 occluders in order to manage even to improve anything at all.