White line around the edge of custom textures

Discussion in 'Mapping Questions & Discussion' started by Blinx, Mar 22, 2016.

  1. Blinx

    Blinx L69: Deviant Member

    Messages:
    69
    Positive Ratings:
    39
    This is more so a problem with Photoshop but it affects my mapping so I thought I'd give a shot asking for help here.

    When creating translucent textures in Photoshop and converting them to VTFs I get this horrible mis-alpha'd edge around all the pixels.

    My process is: Ctrl+click all the layers in PS -> create alpha channel -> save as 32-bit TGA. is there a different method I should use? I'm using PS 5.1 if that's important.

    Thanks in advance.
     
  2. Nicky

    aa Nicky Lets try something new!

    Messages:
    912
    Positive Ratings:
    1,699
    Have you tried it in game? Or are you just looking at the preview in VTF Edit? The preview, and in the texture browser in hammer can make the transparent parts look weird (they can look weird in different ways on different images), but they are actually okay in game.

    In Texture Browser (and similar in the preview in vtf edit):
    Capture.PNG

    When I put the texture on a brush:
    Capture2.PNG

    Also make sure that you have your vmt file set up correctly:
    Capture3.PNG
     
  3. Blinx

    Blinx L69: Deviant Member

    Messages:
    69
    Positive Ratings:
    39
    FPPS.JPG FPIG.JPG
    VMT:

    Code:
    "LightmappedGeneric"
    {
        "$basetexture" "mapmaterials/general/frogportal"
        "$translucent" 1
        "$decal" 1
        "$decalscale" 0.25
    }
    
    It's an observation from both PS and in-game, I understand what's happening, it's a matter of how do I get photoshop to properly select anti-aliased edges.
     
  4. MaccyF

    aa MaccyF Notoriously Unreliable

    Messages:
    911
    Positive Ratings:
    1,474
    your colour should extend into the alpha masked zone, select-modify-expand your alpha mask a couple of pixels and fill it black
     
  5. Blinx

    Blinx L69: Deviant Member

    Messages:
    69
    Positive Ratings:
    39
    Yes but how do I do this easily? I could go around all the edges and make the colour bleed out or push the alpha mask in but that's not exactly a process I want to go through each time I want to make a translucent texture.
     
  6. MaccyF

    aa MaccyF Notoriously Unreliable

    Messages:
    911
    Positive Ratings:
    1,474
    it takes literally 5 clicks, select your alpha mask, select-modify selection-expand seclectrion-fill new selection
     
    Last edited: Mar 27, 2016
  7. Blinx

    Blinx L69: Deviant Member

    Messages:
    69
    Positive Ratings:
    39
    frogportal2.JPG
    This is the effect I get in-game if I use that method, it's not desirable, it's eating into actual pixels and doesn't even fix it in a lot of places.
     
  8. Nicky

    aa Nicky Lets try something new!

    Messages:
    912
    Positive Ratings:
    1,699
    I usually just use paint.net and manually erase the parts that are supposed to be transparent. Obviously this isn't very efficient, but I don't own PS and I've been manually doing it for over a year now that it's super quick and easy.
     
    • Agree Agree x 1
  9. Lampenpam

    aa Lampenpam

    Messages:
    1,019
    Positive Ratings:
    337
    https://developer.valvesoftware.com/wiki/$translucent

    So the issue you have is when alpha goes from 100% translucent to 0% translucent at the next pixel. (pretty amateurish explaination I guess)
    Try giving your font a transparent outline, or just a subtle black shadow.
     
  10. Idolon

    aa Idolon the worst admin

    Messages:
    1,554
    Positive Ratings:
    4,462
    The page Lampempam linked has a good explanation, though it could use some further explanation (I didn't understand it for a while myself).

    The issue is that your image format isn't storing color data for fully transparent pixels. Mipmapping/filtering/compression/etc. then blends together your colored pixel with the fully transparent, colorless pixel, and it ends up as a semi-transparent semi-colorless pixel (hence, the white outline).

    The fix for this is to use an image format/image editing program that is capable of storing color data for fully transparent pixels. In my experience, Paint.net is not capable of dealing with this (or, at the very least, doesn't handle it very well), and you'll need to use the GIMP or Photoshop.

    Example:
    [​IMG]

    This is how a texture I made appears in Hammer's texture preview. Texture preview doesn't show the alpha channel, so it looks like a bunch of scribbles. However, applied as an overlay:
    [​IMG]

    Because the color part of the image continues beyond where the alpha channel goes fully transparent, the fuzzy edges blend with the appropriate color instead of white.

    You can possibly get away with this in Paint.net by using an alpha of 1. Since the pixels are still technically visible, the image should keep the color data, and you'll get appropriate blending.

    Hope this helps!
     
    • Thanks Thanks x 1
    • Useful Useful x 1
  11. Pocket

    aa Pocket func_croc

    Messages:
    4,489
    Positive Ratings:
    2,219
    For what it's worth, because Hammer's texture browser STILL can't read alpha channels, Valve often makes the text a slightly different color from the background (the DATA LINK sign is a good example), enough to make it readable in the texture browser but not enough to affect how it looks when placed.
     
    • Thanks Thanks x 1
  12. Blinx

    Blinx L69: Deviant Member

    Messages:
    69
    Positive Ratings:
    39
    Thanks for the advice guys but I think I should have made it clearer what exactly the problem is since I think this got a little detoured into something different, I've come up with a solution which is to simply save as PNG rather than TGA, it's not perfect but it looks better
     
  13. Lampenpam

    aa Lampenpam

    Messages:
    1,019
    Positive Ratings:
    337
    just give your texture a very subtle shadow and it will be perfect
     
  14. UKCS-Alias

    aa UKCS-Alias Mann vs Machine... or... Mapper vs Meta?

    Messages:
    1,264
    Positive Ratings:
    748
    paint.net doesnt handle it well without plugins. It makes the back layer 1 color (black or white). It only does the pixels fine for semi transparent parts. There probably is a plugin that fixes the issue, but other than that it does a poor job there.

    However, class icons for mvm do not need compression and for this it works fine.
     
  15. Tumbolisu

    aa Tumbolisu  I ⌄ I 

    Messages:
    949
    Positive Ratings:
    936
    I have never experienced any alpha problems with paint.net, and it's also the only program I use. I always heard people complain at GIMP for making fully transparent pixels black.
     
  16. henke37

    aa henke37

    Messages:
    1,919
    Positive Ratings:
    449
    Strange, since gimp has the option to store color for completely transparent pixels.
     
  17. hutty

    aa hutty

    Messages:
    471
    Positive Ratings:
    375
    Unfortunately its true, gimp will store 0% transparency pixels as black, so you would need to bump it up to 1% transparency.
     
  18. takabuschik

    aa takabuschik

    Messages:
    663
    Positive Ratings:
    316
    I have to disagree. I once had Photoshop (cs6), and I hated the .psd to .vtf plugin. I also didn't like the whole alpha channel thing, it made everything more complicated. Now I use Paint.net only (lost the photoshop program, it was on my old computer, which broke down). I have no plugins installed, but still everything is just fine.
    Paint.net has the alpha channel built-in each layer, the .vtf plugin is much more friendly, and I haver had any issues with transparency what so ever.
    It's obvious that Photoshop is better than paint.net, but in the price of photoshop (at least where I live), I could by a nice little used car. And considering paint.net doesn't have any issues with making textures (nothing noticable at least), it's clearly the way to go.
     
  19. Idolon

    aa Idolon the worst admin

    Messages:
    1,554
    Positive Ratings:
    4,462
    It's worth noting that I've never used Paint.net's .vtf plugin, which leads me to believe that it stores color data for transparent pixels in memory, but does not save them to .png or .tga.
     
  20. hutty

    aa hutty

    Messages:
    471
    Positive Ratings:
    375
    I found this http://graphicdesign.stackexchange....-transparent-color-other-than-black-with-gimp
    Turns out gimp will respect colors of 0% alpha as long as you stick to one layer.

    And I have to agree with you on photoshop, everything simple is twice as complicated as it needs to be, or buried in a menu location that makes no sense.
    If you need a photoshop plugin but cannot afford CS6 or CC, the registration servers for CS2 are no longer active. So you can download it from adobe's site, toss a placeholder key, and get to work.
    Details here
    http://www.redmondpie.com/download-adobe-photoshop-cs2-for-free-legally-while-you-still-can/
    sorry if the link text is clickbaity
     
    • Agree Agree x 1