Writing a Pakrat replacement

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
Edit: The final product exists and is ready for use. For more information, please visit the Project Website.

=== Original Post Follows ===
So this evening I started working in a replacement system for Pakrat that wasn't as onerous as plain bspzip.

I've had some bad experiences (read: corrupted contents) with Pakrat: While handy-looking, it seems to be an abandoned project, (three years without an update) and I'm not sure I'd be able to bring it into the present even if the author gave me the source code. (I could decompile it, but...)

The design is this: A python script will tie together bspzip, vbspinfo, and hlextract, in order to automatically determine which models/textures in a map are provided by the TF2 gcfs and which ones have to be packed in. This approach has the advantage of using Valve's own official functions for BSP packing and interpretation, and avoids reinventing the wheel of GCF inspection from HlLib.

Does this sound useful to folks? Are there existing shortcuts I'm missing? Has this already been done and I just don't know about it?
 
Last edited:

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
After about two hours of work I've got it to the stage where it makes mostly-correct determinations about what textures/models in my BSP are not present in the GCF files.

What GCFs should I be inspecting for TF2? Right now it's collecting data from:
  • source materials.gcf
  • source models.gcf
  • source 2007 shared materials.gcf
  • fortress 2 client content.gcf
  • fortress 2 content.gcf
  • team fortress 2 materials.gcf

I'm a little confused in that some materials seem to be in the GCF path <ROOT>/hl2/materials/ but are still valid since they're in a shared GCF. Should I just ignore the first "game" folder in most GCFs as long as I know they are ones present on all TF2-installed systems?

Is there some file I can interpret to determine what GCFs are needed for a particular game?

(Holy speculative questioning! Tune in for the thrilling conclusion, same bat-channel, same bat-time!)
 
Last edited:

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
As long as you don't expect to get them too soon, since I started the actual programming part just today :p

Things that sound cool but definitely won't be in the first cut...
  • Detecting soundscapes
  • Parsing soundscapes to determine custom .wav content
  • Detected particle manifests
  • Parsing particle manifests to determine custom .pcf content
  • Parsing non-GCF'ed .vmt files to know what .vtf files may need packing
  • Making it easy to slot in a config for a mod other than TF2
  • .ain navgraphs (not too useful in TF2 without bots)
  • Detecting assets which are both on the disk (tf/ directory) but also in the GCFs, testing if they are different, prompting user to decide whether it means including or not.
  • Extracting things from the GCF or BSP and passing them on to an external viewing program
  • Map briefing files / Intro video

I'm not planning any sort of graphical GUI. I had enough pain the last time I worked with WxPython. (Frankly if I did do a GUI I'd be doing this in Java.) Instead I want to focus on something that's easy to chain in to a scripted workflow.
 
Last edited:

Nutomic

L11: Posh Member
Feb 7, 2009
888
177
Actually, something like this already exists, and it also works for all source games. Its called "map analyst", download it here

Only thing you will still need to pakrat are things like the mission description and images for it, if you need those.
 

Nutomic

L11: Posh Member
Feb 7, 2009
888
177
I dont really understand what you need a GUI for, you just put some lines in the advanced compile options and thats it, nothing which would require a GUI.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,135
6,056
I dont really understand what you need a GUI for, you just put some lines in the advanced compile options and thats it, nothing which would require a GUI.

Except removing all the duplicated things I get added by default and adding in the extra stuff.. This is the 21st century, I dont want to be altering things on a command line basis!!
 

A Boojum Snark

Toraipoddodezain Mazahabado
aa
Nov 2, 2007
4,775
7,669
Detecting assets which are both on the disk (tf/ directory) but also in the GCFs, testing if they are different, prompting user to decide whether it means including or not.
No. If it's in the cache you do. not. embed.

Also, while everyone is going zomgui I'd like to say: this would require the python base stuff installed right? It's not exactly a standard thing to have (like java and .net have become) so uhh... I don't know what I really wanted to say. Just that you'll probably end up where 95% of your customers need to go get that too.
 

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
When I say "no GUI" I was thinking of either
  • A text interface that's still interactive ("type quit to exit")
  • A frontend mouse-around-GUI frontend as a separate application that controls the command-backend. Like how VBCT is a "frontend" for vbsp/vvis/vrad.

I originally considered making (and indeed, started coding) the whole thing in Java, so if an interactive mouse-driven GUI is such a must-have it's certainly less-painful than Python GUIs.

Except removing all the duplicated things I get added by default and adding in the extra stuff.. This is the 21st century, I dont want to be altering things on a command line basis!!

Duplicated things? Extra stuff? Could you elaborate on this problem/frustration?

Part of the reason I was thinking of omitting a pakrat-esque GUI is because it seems like most of the important bits (textures, models, soundscapes, particles) can all be done automatically, and the rest would be relatively simple prompts.

No. If it's in the cache you do. not. embed.
Not with the same name, of course. What I mean is it seems like a place where some sort of action should be taken or alert raised, potentially because the mapper may have never had an accurate view of his own map due to local modifications.

Also, while everyone is going zomgui I'd like to say: this would require the python base stuff installed right? It's not exactly a standard thing to have (like java and .net have become)
AFAIK Java is not installed on Windows by default, someone has to download it too. I'd love it if it was (and not Microsoft's "embrance and corrupt" strategy) since it would shut up the people who complain that they have to download it to run things.
 
Last edited:

Balloonsfor600

L1: Registered
Sep 7, 2009
45
18
Hmmm... If it's easy to use without the GUI I say why do you need it (even though I would prefer one)? Long story short, I will keep an eye on this.

In the meantime - I think that I will try this Map Analyst that Nutomic has described.
 

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
Can anyone provide example cases where automation breaks down and a GUI (one more complex than Yes/No/Cancel) is required? I'm working under the assumption that the map already plays correctly on the packager's computer, so all the custom models and textures etc. are already in the tf/ directory on-disk.

I'm not saying they don't exist, but for design purposes I'd like to figure out the use cases.
 
Last edited:

Freyja

aa
Jul 31, 2009
2,994
5,813
When I fisrt used Pakrat it tooks me ages to find the right god damn thing to install on that horrible Java site.

Why can't things like that be included on the windows DVD as an "optional install" like mac has the dev tools?
 

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
Finding what to install for Java is so much easier now.

That said, it's not included along with the DVD because it's from a separate company from Microsoft, and Microsoft has their own somewhat competing set of tools. And the last time Microsoft did something like that they basically riddled the installed version with Windows-specific features in order to break the spec (analogous to what Internet Explorer did for the quality of HTML :cursing: ) and ended up paying $20 million in a lawsuit.