Distributed map compiling: Needs testing!

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
For SCIENCE!

http://sentry.tf2stats.net/
Client updated 1/22/10 @ 6:00 EST. Please update if you are running one of the older versions

Instructions, files, and stuff is there. This program will basically harness other machines on your network to make your compiles faster. It will ONLY work with machines inside your LAN.

Right now the system is pretty experimental, but I'd like some info from anyone who tests it:
- Map compile time WITHOUT Sentry
- Map compile time WITH Sentry, as well as the number of worker machines you had.
- Any anomalies in the final map's compile (Strange lighting, etc. Anything that looks different from a normal compile.

Compile times:
Hydro without Sentry: 9 minutes 58 seconds
Hydro WITH Sentry, VRAD only, 2 workers: 7 minutes 4 seconds
Hydro with sentry, VRAD + VVIS, 2 workers: 4 minutes 53 seconds.
Post yours please :3
 
Last edited:

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
What? I was trying VMPI earlier this week and couldn't get the worker to be recognized.

I thought VMPI for VRAD (the longest phase for me) was broken for the Orange Box engine part of the SDK. Did it get fixed, or is Sentry doing some sort of proxying-fix deal?
 
Last edited:

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
What? I was trying VMPI earlier this week and couldn't get the worker to be recognized.

I thought VMPI for VRAD (the longest phase for me) was broken for the Orange Box engine part of the SDK. Did it get fixed, or is Sentry doing some sort of proxying-fix deal?

Two things.

One, you'll notice you need a special version of VRAD (It's in the instructions). Second, I spent the last few days tweaking around with the worker options (Many of them are hidden! Tricksy valve) until i got something that worked. It seems with OB, you need to run the workers in SDK mode, but force it to use the non-SDK packet distributor. :facepalm:
 

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
<3 Now make a mac client so I can harness my mac and we're good to go?
This would require mac builds of vvis, vrad, and some other misc bits. Unlikely. :(

You can try crossover though.

> What is a grue?
The grue is a sinister, lurking presence in the dark places of the earth. Its favorite diet is adventurers, but its insatiable appetite is tempered by its fear of light. No grue has ever been seen by the light of day, and few have survived its fearsome jaws to tell the tale.
 

Terr

Cranky Coder
aa
Jul 31, 2009
1,590
410
Ah, you built a new one from source. Verrahnice. So basically they forgot to change a switch in the file to use the "Works outside of our LAN" network setup?

I'd suggest sending in the diffs/patchfiles/diagnosis to Valve... might make them fix the stock stuff faster.
 
Last edited:

Cameron:D

L6: Sharp Member
Sep 28, 2008
363
145
Ok, I'm trying this out.

I have 3 computers:
lil-lappy
server
Main

I'm compiling one of the Valve maps included with the SDK (sdk_pl_goldrush). When the VMPI window came up I added it to the queue, and now it seems nothing is happening :(

EDIT: Its been like 20 mins, nothing. Cancelling.
Will try again tomorrow.
 
Last edited:

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
I'm compiling one of the Valve maps included with the SDK (sdk_pl_goldrush). When the VMPI window came up I added it to the queue, and now it seems nothing is happening :(

EDIT: Its been like 20 mins, nothing. Cancelling.
Will try again tomorrow.

Hmm. that's strange. The setup looks good from here, and it still works for me.

Grab me on steam some time today. We need to find out if my instructions are bad, or my code is bad.
 

Nutomic

L11: Posh Member
Feb 7, 2009
888
177
Submit Job and About give a 404 error, doesnt look too good.

I dont have multiple pcs, so i cant test it, but i would love something that works online (no idea if its possible).
 

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
Submit Job and About give a 404 error, doesnt look too good.

I dont have multiple pcs, so i cant test it, but i would love something that works online (no idea if its possible).

Oh yeah dont' click the tabs. Site's totally not done. it's beta, remember?;)

The submit job section is on the main page, just scroll down.

Matt and I tested online compiling a lot. However, the SDK distributor is totally busted (Which is more optimized for higher latency internet compiles), and you'd need something like a 10mbit uplink to get much out of it.
 

Cameron:D

L6: Sharp Member
Sep 28, 2008
363
145
Alright, I tried some compiling today...
VVIS ran fine, but it gave a problem about having a different exe name for VRAD. I put the custom one into the orangebox/bin folder.
 

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
Alright, I tried some compiling today...
VVIS ran fine, but it gave a problem about having a different exe name for VRAD. I put the custom one into the orangebox/bin folder.

Did you specify that EXE when you built the map? If building from hammer, it's tools -> options -> build programs, select vrad_LAA.exe. If you're not building with hammer, you'll have to figure out how to do this on your own; but most build environments should support changing the vrad exe.

Or, if the error you got was a mismatch between vrad_LAA and vvis, then I need to tweak the delays a bit. But in the mean time you can simply re-submit the job.
 

Ravidge

Grand Vizier
aa
May 14, 2008
1,544
2,818
Why does it need input on a webpage to work?

Anyway, I have no idea how to set this up. Write a step by step instruction, or even better a video. Looking forward to it.
 

Ezekel

L11: Posh Member
Dec 16, 2008
818
245
nice, you've found a way to mimic the UDK's swarm system in source :)
are subtractive brushes going to be next? (imagine being able to use subtractive brushes in hammer... could get some truly good timesaving here for complex shapes XD ).


step by step guid like ravidge said would be nice.
 

Cameron:D

L6: Sharp Member
Sep 28, 2008
363
145
Why does it need input on a webpage to work?

Anyway, I have no idea how to set this up. Write a step by step instruction, or even better a video. Looking forward to it.

1. Download the modified vrad (vrad_LAA.exe), put this in the %sourcesdk%\bin\orangebox\bin\ folder.

2. Open hammer, go into the hammer options, to the build programs tab, and change RAD executable to
Code:
$SteamUserDir\sourcesdk\bin\orangebox\bin\vrad_LAA.exe

3. Download the Sentry client on each computer you want to have compiling (Local network only), and run it. It should say something like "Sent connection string" in the console.

4. Check the Sentry Website to see if your computers show up (in the workers section). If they don't show up, check your firewall etc (or the sentry server is bugged out, as it seems to be now)

5. Start compiling your map (Not -final). After VBSP finishes, a VMPI
tracker window should open, now go to the website and enter the details into the start job section.
Enter the name of the map (Seems to work both with and without the .bsp extension)
IP Address IP should be your external IP, default is correct.
The Local IP should be set to the ip of the 'host' computer - the one with the VMPI tracker window open
Leave port as-is.

6. Sit and stare at the VMPI tracker window as all the pretty colours flash everywhere :>
 

FireSlash

L2: Junior Member
Sep 29, 2009
50
49
(or the sentry server is bugged out, as it seems to be now)

I restarted the server, is it still doing this?

Why does it need input on a webpage to work?

Anyway, I have no idea how to set this up. Write a step by step instruction, or even better a video. Looking forward to it.
This is temporary. I'll write a script that can automate the process once i know it actually works. But I'm lacking (any) test data at the moment.
 
Last edited:

chickenm4n

L1: Registered
Nov 27, 2009
49
6
quick! someone create a program that fools this program into thinking that the INTERNET is a lan connection!

withe power of 4 billion computers at your compiling finger tips...
 

Cameron:D

L6: Sharp Member
Sep 28, 2008
363
145
PROTIP: They need to be running the client :)
I also had the thought of writing my own project for BOINC that will get a map and run the compile process on it, but it would probably take too long get the compiled map back to be worthwhile.