[Tutorial] REMOVING UGLY SHADOWS

YM

LVL100 YM
aa
Dec 5, 2007
7,158
6,079
wow, great tutorial youme
i just wonder, who is your go-to-guy if you run into problems? robin walker? ;)

No I just complain and curse loudly at hammer until it works. I found out about the special texture lighting ages back when this guy wrote a blog about them - here - and I was thinking about why goldrush had nice shadows on its tracks when 100% of custom payload maps either had horrible shadows or none at all for their tracks, and if you remember I complained a lot and had lots of comments on it whilst working on pl_hoodoo. Then when talking with a friend via steam we figured that the compile options in that blog might help, so we both set up some test maps and he knew a little more about using .rad files than I did so he filled me in on how to set them up and between us we discovered why valve maps look so much better than ours.

I think about things for a long time if they dont work, I read up on possible entity solutions in the VDC and look at how other maps have done what I want to to figure stuff out. Because with things like this, there isn't anyone you can ask, we're at the very top, like when goldrush came out there is no one to ask you just have to get your head down and figure it out for yourself.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,158
6,079
Further explaination of -staticproplighting:

OK, so I've mocked up a little scene that might happen in any TF2 map, I've used the corrugated metal sheets because they illustrate what -staticproplighting does well because they have a high poly desnsity for their size as well as the fact they are a common problem for lighting and something people often get wrong when finding a fix.
My scene looks like this:
3170966392_0329fcf328_o.jpg



So here's what it looks like ingame, not very good is it?
3170965956_e55da70483_o.jpg

So why does it look so terrible?
well, The two metal sheets on the sides are both facing outwards and lighting is taken from their origin. These two things combined makes sure they're black in this instance. The front one's origin is right behind the wooden beam so the light is being blocked, making it dark. The back one's origin is facing away from the light source and since it's only a one sided prop it gets no light on it.

So what can we do about it?
Well the first thing to do is to make sure all the origins are facing the lightsource so that they get light falling on them. Here it is again with the far sheet rotated 180 so the origin is facing the other way (you can tell in hammer which way it's facing because there is a little yellow line once you select the prop, the direction of that line tells you which way it's facing)
3170965890_55156dc688_o.jpg


But you can see there that whilst we've fixed the back sheet the front sheet is still dark even though its origin is facing the light. We could use an info_lighting (info_target actually since info_lighting is broken) to 'move' it's origin so that it is in the light, but we can do one better than that.

In the first post there are three extra commands, -staticproppolys -staticproplighting and -textureshadows. The first and the last deal with taking shadows from the props exact outline rather than the collision mesh (great for props with no collision mesh like the payload tracks!) and taking shadows from props with alpha textures(great for chainlink fences). The middle one, -staticproplighting works out the lighting on props based on each vertex instead of the origin, so one side of a prop might be in light and lit nicely whilst the other side might have the light blocked to it so it will be dark.

Here is what it looks like with -staticproplighting on:
3170136301_a1cce75411_o.jpg

As you can see that looks much better because the metal sheets are now light where the light hits them and darker where the wood is blocking the light and casting shadows. Admittedly the resolution is a little low because the points at which lighting is done are fairly far apart and the dark patches are quite dark (that's because I don't have much ambient light in this scene) but I think you'll agree it looks much better than the first or second compile.

Here it is again showing the back of the front panel, it shows how the wood blocks the light from both sides of the sheet. and how the panel on the floor has a shadow across it in the same place the ground would if the metal sheet were not there.
3170966092_a276ee4568_o.jpg


This method will also help any props that have their origin embedded in geometry, since all the prop that is outside of the geometry will be lit properly - great for those large rocks which usually have their origin below a displacement.
 
Last edited:
May 23, 2009
205
34
Code:
forcetextureshadow models/props_gameplay/security_fence_big01.mdl
forcetextureshadow models/props_gameplay/security_fence_big02.mdl
forcetextureshadow models/props_gameplay/security_fence512.mdl
forcetextureshadow models/props_gameplay/security_fence512_lower.mdl
forcetextureshadow models/props_gameplay/security_fence512_skybox.mdl
forcetextureshadow models/props_gameplay/security_fence256.mdl
forcetextureshadow models/props_gameplay/security_fence32.mdl
forcetextureshadow models/props_gameplay/security_fence64.mdl
forcetextureshadow models/props_gameplay/security_fence64_lower.mdl
forcetextureshadow models/props_gameplay/security_fence80.mdl
forcetextureshadow models/props_gameplay/security_fence_section01.mdl
forcetextureshadow models/props_gameplay/security_fence128.mdl
forcetextureshadow models/props_gameplay/security_fence128_lower.mdl
forcetextureshadow models/props_gameplay/security_fence256_gate01.mdl
forcetextureshadow models/props_gameplay/security_fence256_gate02.mdl
forcetextureshadow models/props_gameplay/security_fence256_lower.mdl
forcetextureshadow models/props_farm/haypile001.mdl
forcetextureshadow models/props_farm/tree001.mdl
forcetextureshadow models/props_farm/tree001_skybox.mdl
forcetextureshadow models/props_gameplay/door_grate001_bottom.mdl
forcetextureshadow models/props_gameplay/door_grate001_top.mdl
forcetextureshadow models/props_gameplay/door_grate002_bottom.mdl
forcetextureshadow models/props_gameplay/door_grate002_top.mdl
forcetextureshadow models/props_gameplay/door_grate003_bottom.mdl
forcetextureshadow models/props_gameplay/door_grate003_top.mdl
forcetextureshadow models/props_foliage/bramble001a.mdl 
forcetextureshadow models/props_foliage/bramble001a_skybox.mdl 
forcetextureshadow models/props_foliage/corn_plant01.mdl
forcetextureshadow models/props_foliage/shrub_01a.mdl
forcetextureshadow models/props_foliage/shrub_02a.mdl
forcetextureshadow models/props_foliage/shrub_03_card.mdl
forcetextureshadow models/props_foliage/shrub_03_card_skybox.mdl
forcetextureshadow models/props_foliage/shrub_03_cluster.mdl
forcetextureshadow models/props_foliage/shrub_03_cluster02.mdl
forcetextureshadow models/props_foliage/shrub_03_cluster_skybox.mdl
forcetextureshadow models/props_foliage/shrub_03a.mdl
forcetextureshadow models/props_foliage/shrub_03b.mdl
forcetextureshadow models/props_foliage/shrub_03c.mdl
forcetextureshadow models/props_foliage/shrub_04a.mdl
forcetextureshadow models/props_foliage/shrub_04b.mdl
forcetextureshadow models/props_foliage/tree01.mdl
forcetextureshadow models/props_foliage/tree02.mdl
forcetextureshadow models/props_foliage/tree_pine01.mdl
forcetextureshadow models/props_foliage/tree_pine01_4cluster.mdl
forcetextureshadow models/props_foliage/tree_pine01_8cluster.mdl
forcetextureshadow models/props_foliage/tree_pine01_8cluster.mdl
forcetextureshadow models/props_forest/shrub_03_cluster.mdl
forcetextureshadow models/props_forest/shrub_03b.mdl
forcetextureshadow models/props_forest/shrub_03c.mdl

Add these new ones for the recent update:
Code:
forcetextureshadow models/props_foliage/tree_pine_huge.mdl
forcetextureshadow models/props_foliage/tree_pine_small.mdl
forcetextureshadow models/props_foliage/tree_pine_extrasmall.mdl[B][/B]
 

Open Blade

L420: High Member
Nov 30, 2007
439
34
Further explaination of -staticproplighting:

OK, so I've mocked up a little scene that might happen in any TF2 map, I've used the corrugated metal sheets because they illustrate what -staticproplighting does well because they have a high poly desnsity for their size as well as the fact they are a common problem for lighting and something people often get wrong when finding a fix.
My scene looks like this:
3170966392_0329fcf328_o.jpg



So here's what it looks like ingame, not very good is it?
3170965956_e55da70483_o.jpg

So why does it look so terrible?
well, The two metal sheets on the sides are both facing outwards and lighting is taken from their origin. These two things combined makes sure they're black in this instance. The front one's origin is right behind the wooden beam so the light is being blocked, making it dark. The back one's origin is facing away from the light source and since it's only a one sided prop it gets no light on it.

So what can we do about it?
Well the first thing to do is to make sure all the origins are facing the lightsource so that they get light falling on them. Here it is again with the far sheet rotated 180 so the origin is facing the other way (you can tell in hammer which way it's facing because there is a little yellow line once you select the prop, the direction of that line tells you which way it's facing)
3170965890_55156dc688_o.jpg


But you can see there that whilst we've fixed the back sheet the front sheet is still dark even though its origin is facing the light. We could use an info_lighting (info_target actually since info_lighting is broken) to 'move' it's origin so that it is in the light, but we can do one better than that.

In the first post there are three extra commands, -staticproppolys -staticproplighting and -textureshadows. The first and the last deal with taking shadows from the props exact outline rather than the collision mesh (great for props with no collision mesh like the payload tracks!) and taking shadows from props with alpha textures(great for chainlink fences). The middle one, -staticproplighting works out the lighting on props based on each vertex instead of the origin, so one side of a prop might be in light and lit nicely whilst the other side might have the light blocked to it so it will be dark.

Here is what it looks like with -staticproplighting on:
3170136301_a1cce75411_o.jpg

As you can see that looks much better because the metal sheets are now light where the light hits them and darker where the wood is blocking the light and casting shadows. Admittedly the resolution is a little low because the points at which lighting is done are fairly far apart and the dark patches are quite dark (that's because I don't have much ambient light in this scene) but I think you'll agree it looks much better than the first or second compile.

Here it is again showing the back of the front panel, it shows how the wood blocks the light from both sides of the sheet. and how the panel on the floor has a shadow across it in the same place the ground would if the metal sheet were not there.
3170966092_a276ee4568_o.jpg


This method will also help any props that have their origin embedded in geometry, since all the prop that is outside of the geometry will be lit properly - great for those large rocks which usually have their origin below a displacement.

So are you saying, when we run a compile using Hammer, in the bottom field "additional game perameters" we should enter these 3 commands just like this, with a space between each one?

-staticproppolys -staticproplighting -textureshadows
 

Penguin

Clinically Diagnosed with Small Mapper's Syndrome
aa
May 21, 2009
2,039
1,484
Okay, I just ran this word for word, but it compiles in about 1/10th the time and seems to decide to ignore my light_env entity.
 

grazr

Old Man Mutant Ninja Turtle
aa
Mar 4, 2008
5,441
3,814
Can we get this stickied? This has to be one of the most important mid-late map version tutorials out here.

I get so many people asking about this. As i'm sure Youme does being the author of this and all.
 

RavenStryker

Former Alias: †Blade†/Xi.Cynx
aa
Nov 25, 2008
782
849
I was just wondering if there was any more updates on this as for what props can be added to the list, i don't know how to find out which ones to add otherwise i'd do it myself. But i was getting ready to compile and thought that maybe since there has been a lot of props added that there has to be some more like the ones on the list already.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,158
6,079
Combined the third page's explaination of staticproppolys into the original post.

Blade you need to add any props that have an alpha channel, foliage etc.
 

Tapp

L10: Glamorous Member
Jan 26, 2009
776
216
I'm pretty sure valve uses this, they developed it in episode 2 to deal with the trees in the forest road sections. Thanks for the tut though, I'll have to finish a map first.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,158
6,079
I'm pretty sure valve uses this, they developed it in episode 2 to deal with the trees in the forest road sections. Thanks for the tut though, I'll have to finish a map first.

You'll find that as of the orange box, valve haven't once not-used it. THIS is the standard for the source engine yet, they just didn't make hammer do them by defult yet.
 

JoshuaC

L420: High Member
Sep 2, 2008
444
164
You'll find that as of the orange box, valve haven't once not-used it. THIS is the standard for the source engine yet, they just didn't make hammer do them by defult yet.

It's not enabled by default merely cause it's meant to be a finishing pass. If I had to sit through a static prop compile each time I compiled my map, I'd probably kill myself.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,158
6,079
It's not enabled by default merely cause it's meant to be a finishing pass. If I had to sit through a static prop compile each time I compiled my map, I'd probably kill myself.

I mean for the "final" options in the expert compile mode. I personally feel it should be in the 'normal' compiles, since you've got 'fast' for testing but then using the expert compile check boxes make it so easy to do a tailored compile it hardly matters
 

Psy

The Imp Queen
aa
Apr 9, 2008
1,706
1,491
Calculating shadows per vertex doesn't take very long. It's casting shadows based on the physical model that takes forever.

Anyway. you should add more info on prop-specific properties that affect how VRAD handles lighting. I'll do it for you. :p

---

Here's your typical set of alpine trees and here is what happen when you compile with -staticproppolies.

vert_shadow1.jpg


Oh noooooo! What is happening here is that each branch is casting shadows onto branches below it which causes the vertices to be lit up horrendously as shown in the picture above.

So how do you fix such a thing? Simples. Go to your prop_static and change the following property to 'Yes'.

vert_shadow3.jpg


As the name implies, the prop will no longer cast shadows onto itself. ;)

Recompile and voila!

vert_shadow2.jpg


All the fun of vertex-based lighting with none of the mess!