Writing a Pakrat replacement

Discussion in 'Mapping Questions & Discussion' started by Terr, Sep 16, 2009.

  1. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    Edit: The final product exists and is ready for use. For more information, please visit the Project Website.

    === Original Post Follows ===
    So this evening I started working in a replacement system for Pakrat that wasn't as onerous as plain bspzip.

    I've had some bad experiences (read: corrupted contents) with Pakrat: While handy-looking, it seems to be an abandoned project, (three years without an update) and I'm not sure I'd be able to bring it into the present even if the author gave me the source code. (I could decompile it, but...)

    The design is this: A python script will tie together bspzip, vbspinfo, and hlextract, in order to automatically determine which models/textures in a map are provided by the TF2 gcfs and which ones have to be packed in. This approach has the advantage of using Valve's own official functions for BSP packing and interpretation, and avoids reinventing the wheel of GCF inspection from HlLib.

    Does this sound useful to folks? Are there existing shortcuts I'm missing? Has this already been done and I just don't know about it?
     
    • Thanks Thanks x 18
    Last edited: Jul 29, 2010
  2. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    After about two hours of work I've got it to the stage where it makes mostly-correct determinations about what textures/models in my BSP are not present in the GCF files.

    What GCFs should I be inspecting for TF2? Right now it's collecting data from:
    • source materials.gcf
    • source models.gcf
    • source 2007 shared materials.gcf
    • fortress 2 client content.gcf
    • fortress 2 content.gcf
    • team fortress 2 materials.gcf

    I'm a little confused in that some materials seem to be in the GCF path <ROOT>/hl2/materials/ but are still valid since they're in a shared GCF. Should I just ignore the first "game" folder in most GCFs as long as I know they are ones present on all TF2-installed systems?

    Is there some file I can interpret to determine what GCFs are needed for a particular game?

    (Holy speculative questioning! Tune in for the thrilling conclusion, same bat-channel, same bat-time!)
     
    Last edited: Sep 16, 2009
  3. TheBladeRoden

    TheBladeRoden L7: Fancy Member

    Messages:
    491
    Positive Ratings:
    167
    are you far enough along that I can start making feature requests?
     
  4. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    As long as you don't expect to get them too soon, since I started the actual programming part just today :p

    Things that sound cool but definitely won't be in the first cut...
    • Detecting soundscapes
    • Parsing soundscapes to determine custom .wav content
    • Detected particle manifests
    • Parsing particle manifests to determine custom .pcf content
    • Parsing non-GCF'ed .vmt files to know what .vtf files may need packing
    • Making it easy to slot in a config for a mod other than TF2
    • .ain navgraphs (not too useful in TF2 without bots)
    • Detecting assets which are both on the disk (tf/ directory) but also in the GCFs, testing if they are different, prompting user to decide whether it means including or not.
    • Extracting things from the GCF or BSP and passing them on to an external viewing program
    • Map briefing files / Intro video

    I'm not planning any sort of graphical GUI. I had enough pain the last time I worked with WxPython. (Frankly if I did do a GUI I'd be doing this in Java.) Instead I want to focus on something that's easy to chain in to a scripted workflow.
     
    Last edited: Sep 16, 2009
  5. TheBladeRoden

    TheBladeRoden L7: Fancy Member

    Messages:
    491
    Positive Ratings:
    167
    Aww, no GUI? I have to remove my thanks now
     
    • Thanks Thanks x 1
  6. Freyja

    aa Freyja ¯\_(ツ)_/¯

    Messages:
    2,879
    Positive Ratings:
    4,970
    Yeah no GUI is why I dont like BSPzip too much.
     
    • Thanks Thanks x 1
  7. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,745
    A GUI is a must sadly.
     
    • Thanks Thanks x 2
  8. Nutomic

    Nutomic L11: Posh Member

    Messages:
    888
    Positive Ratings:
    177
    Actually, something like this already exists, and it also works for all source games. Its called "map analyst", download it here

    Only thing you will still need to pakrat are things like the mission description and images for it, if you need those.
     
  9. The Political Gamer

    aa The Political Gamer

    Messages:
    4,468
    Positive Ratings:
    1,684
    Um ya, GUI or no download.
     
  10. Nutomic

    Nutomic L11: Posh Member

    Messages:
    888
    Positive Ratings:
    177
    I dont really understand what you need a GUI for, you just put some lines in the advanced compile options and thats it, nothing which would require a GUI.
     
  11. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,745
    Except removing all the duplicated things I get added by default and adding in the extra stuff.. This is the 21st century, I dont want to be altering things on a command line basis!!
     
    • Thanks Thanks x 2
  12. Psy

    aa Psy The Imp Queen

    Messages:
    1,705
    Positive Ratings:
    1,468
    Don't do a Valve. Make a GUI for god's sake.
     
    • Thanks Thanks x 2
  13. A Boojum Snark

    aa A Boojum Snark Toraipoddodezain Mazahabado

    Messages:
    4,769
    Positive Ratings:
    5,535
    No. If it's in the cache you do. not. embed.

    Also, while everyone is going zomgui I'd like to say: this would require the python base stuff installed right? It's not exactly a standard thing to have (like java and .net have become) so uhh... I don't know what I really wanted to say. Just that you'll probably end up where 95% of your customers need to go get that too.
     
  14. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    When I say "no GUI" I was thinking of either
    • A text interface that's still interactive ("type quit to exit")
    • A frontend mouse-around-GUI frontend as a separate application that controls the command-backend. Like how VBCT is a "frontend" for vbsp/vvis/vrad.

    I originally considered making (and indeed, started coding) the whole thing in Java, so if an interactive mouse-driven GUI is such a must-have it's certainly less-painful than Python GUIs.

    Duplicated things? Extra stuff? Could you elaborate on this problem/frustration?

    Part of the reason I was thinking of omitting a pakrat-esque GUI is because it seems like most of the important bits (textures, models, soundscapes, particles) can all be done automatically, and the rest would be relatively simple prompts.

    Not with the same name, of course. What I mean is it seems like a place where some sort of action should be taken or alert raised, potentially because the mapper may have never had an accurate view of his own map due to local modifications.

    AFAIK Java is not installed on Windows by default, someone has to download it too. I'd love it if it was (and not Microsoft's "embrance and corrupt" strategy) since it would shut up the people who complain that they have to download it to run things.
     
    Last edited: Sep 16, 2009
  15. Balloonsfor600

    Balloonsfor600 L1: Registered

    Messages:
    45
    Positive Ratings:
    18
    Hmmm... If it's easy to use without the GUI I say why do you need it (even though I would prefer one)? Long story short, I will keep an eye on this.

    In the meantime - I think that I will try this Map Analyst that Nutomic has described.
     
  16. A Boojum Snark

    aa A Boojum Snark Toraipoddodezain Mazahabado

    Messages:
    4,769
    Positive Ratings:
    5,535
    I didn't mean by default, I meant standard, like I said. Most people have java nowadays.
     
  17. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    Can anyone provide example cases where automation breaks down and a GUI (one more complex than Yes/No/Cancel) is required? I'm working under the assumption that the map already plays correctly on the packager's computer, so all the custom models and textures etc. are already in the tf/ directory on-disk.

    I'm not saying they don't exist, but for design purposes I'd like to figure out the use cases.
     
    Last edited: Sep 16, 2009
  18. Freyja

    aa Freyja ¯\_(ツ)_/¯

    Messages:
    2,879
    Positive Ratings:
    4,970
    When I fisrt used Pakrat it tooks me ages to find the right god damn thing to install on that horrible Java site.

    Why can't things like that be included on the windows DVD as an "optional install" like mac has the dev tools?
     
  19. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    Finding what to install for Java is so much easier now.

    That said, it's not included along with the DVD because it's from a separate company from Microsoft, and Microsoft has their own somewhat competing set of tools. And the last time Microsoft did something like that they basically riddled the installed version with Windows-specific features in order to break the spec (analogous to what Internet Explorer did for the quality of HTML :cursing: ) and ended up paying $20 million in a lawsuit.
     
  20. The Political Gamer

    aa The Political Gamer

    Messages:
    4,468
    Positive Ratings:
    1,684