White line around the edge of custom textures

Blinx

L69: Deviant Member
Mar 14, 2016
69
51
This is more so a problem with Photoshop but it affects my mapping so I thought I'd give a shot asking for help here.

When creating translucent textures in Photoshop and converting them to VTFs I get this horrible mis-alpha'd edge around all the pixels.

My process is: Ctrl+click all the layers in PS -> create alpha channel -> save as 32-bit TGA. is there a different method I should use? I'm using PS 5.1 if that's important.

Thanks in advance.
 

nickybakes

You should've played Rumbleverse
aa
Jul 28, 2015
912
1,741
Have you tried it in game? Or are you just looking at the preview in VTF Edit? The preview, and in the texture browser in hammer can make the transparent parts look weird (they can look weird in different ways on different images), but they are actually okay in game.

In Texture Browser (and similar in the preview in vtf edit):
Capture.PNG


When I put the texture on a brush:
Capture2.PNG


Also make sure that you have your vmt file set up correctly:
Capture3.PNG
 

Blinx

L69: Deviant Member
Mar 14, 2016
69
51
FPPS.JPG
FPIG.JPG

VMT:

Code:
"LightmappedGeneric"
{
    "$basetexture" "mapmaterials/general/frogportal"
    "$translucent" 1
    "$decal" 1
    "$decalscale" 0.25
}

It's an observation from both PS and in-game, I understand what's happening, it's a matter of how do I get photoshop to properly select anti-aliased edges.
 

Blinx

L69: Deviant Member
Mar 14, 2016
69
51
your colour should extend into the alpha masked zone, select-modify-expand your alpha mask a couple of pixels and fill it black

Yes but how do I do this easily? I could go around all the edges and make the colour bleed out or push the alpha mask in but that's not exactly a process I want to go through each time I want to make a translucent texture.
 

Blinx

L69: Deviant Member
Mar 14, 2016
69
51
frogportal2.JPG

This is the effect I get in-game if I use that method, it's not desirable, it's eating into actual pixels and doesn't even fix it in a lot of places.
 

nickybakes

You should've played Rumbleverse
aa
Jul 28, 2015
912
1,741
I usually just use paint.net and manually erase the parts that are supposed to be transparent. Obviously this isn't very efficient, but I don't own PS and I've been manually doing it for over a year now that it's super quick and easy.
 

Idolon

they/them
aa
Feb 7, 2008
2,123
6,137
The page Lampempam linked has a good explanation, though it could use some further explanation (I didn't understand it for a while myself).

The issue is that your image format isn't storing color data for fully transparent pixels. Mipmapping/filtering/compression/etc. then blends together your colored pixel with the fully transparent, colorless pixel, and it ends up as a semi-transparent semi-colorless pixel (hence, the white outline).

The fix for this is to use an image format/image editing program that is capable of storing color data for fully transparent pixels. In my experience, Paint.net is not capable of dealing with this (or, at the very least, doesn't handle it very well), and you'll need to use the GIMP or Photoshop.

Example:
hIPUynS.png


This is how a texture I made appears in Hammer's texture preview. Texture preview doesn't show the alpha channel, so it looks like a bunch of scribbles. However, applied as an overlay:
kM4TzkG.png


Because the color part of the image continues beyond where the alpha channel goes fully transparent, the fuzzy edges blend with the appropriate color instead of white.

You can possibly get away with this in Paint.net by using an alpha of 1. Since the pixels are still technically visible, the image should keep the color data, and you'll get appropriate blending.

Hope this helps!
 

Pocket

Half a Lambert is better than one.
aa
Nov 14, 2009
4,701
2,583
For what it's worth, because Hammer's texture browser STILL can't read alpha channels, Valve often makes the text a slightly different color from the background (the DATA LINK sign is a good example), enough to make it readable in the texture browser but not enough to affect how it looks when placed.
 

Blinx

L69: Deviant Member
Mar 14, 2016
69
51
Thanks for the advice guys but I think I should have made it clearer what exactly the problem is since I think this got a little detoured into something different, I've come up with a solution which is to simply save as PNG rather than TGA, it's not perfect but it looks better
 

UKCS-Alias

Mann vs Machine... or... Mapper vs Meta?
aa
Sep 8, 2008
1,264
817
The fix for this is to use an image format/image editing program that is capable of storing color data for fully transparent pixels. In my experience, Paint.net is not capable of dealing with this (or, at the very least, doesn't handle it very well), and you'll need to use the GIMP or Photoshop.
paint.net doesnt handle it well without plugins. It makes the back layer 1 color (black or white). It only does the pixels fine for semi transparent parts. There probably is a plugin that fixes the issue, but other than that it does a poor job there.

However, class icons for mvm do not need compression and for this it works fine.
 

Tumby

aa
May 12, 2013
1,087
1,196
I have never experienced any alpha problems with paint.net, and it's also the only program I use. I always heard people complain at GIMP for making fully transparent pixels black.
 

henke37

aa
Sep 23, 2011
2,075
515
Strange, since gimp has the option to store color for completely transparent pixels.
 

hutty

aa
Mar 30, 2014
547
446
Unfortunately its true, gimp will store 0% transparency pixels as black, so you would need to bump it up to 1% transparency.
 
Apr 14, 2013
662
344
paint.net doesnt handle it well without plugins. It makes the back layer 1 color (black or white). It only does the pixels fine for semi transparent parts.

I have to disagree. I once had Photoshop (cs6), and I hated the .psd to .vtf plugin. I also didn't like the whole alpha channel thing, it made everything more complicated. Now I use Paint.net only (lost the photoshop program, it was on my old computer, which broke down). I have no plugins installed, but still everything is just fine.
Paint.net has the alpha channel built-in each layer, the .vtf plugin is much more friendly, and I haver had any issues with transparency what so ever.
It's obvious that Photoshop is better than paint.net, but in the price of photoshop (at least where I live), I could by a nice little used car. And considering paint.net doesn't have any issues with making textures (nothing noticable at least), it's clearly the way to go.
 

Idolon

they/them
aa
Feb 7, 2008
2,123
6,137
It's worth noting that I've never used Paint.net's .vtf plugin, which leads me to believe that it stores color data for transparent pixels in memory, but does not save them to .png or .tga.
 

hutty

aa
Mar 30, 2014
547
446
I found this http://graphicdesign.stackexchange....-transparent-color-other-than-black-with-gimp
Turns out gimp will respect colors of 0% alpha as long as you stick to one layer.

And I have to agree with you on photoshop, everything simple is twice as complicated as it needs to be, or buried in a menu location that makes no sense.
If you need a photoshop plugin but cannot afford CS6 or CC, the registration servers for CS2 are no longer active. So you can download it from adobe's site, toss a placeholder key, and get to work.
Details here
http://www.redmondpie.com/download-adobe-photoshop-cs2-for-free-legally-while-you-still-can/
sorry if the link text is clickbaity