Distributed map compiling: Needs testing!

Discussion in 'Mapping Questions & Discussion' started by FireSlash, Jan 22, 2010.

  1. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    For SCIENCE!

    http://sentry.tf2stats.net/
    Client updated 1/22/10 @ 6:00 EST. Please update if you are running one of the older versions

    Instructions, files, and stuff is there. This program will basically harness other machines on your network to make your compiles faster. It will ONLY work with machines inside your LAN.

    Right now the system is pretty experimental, but I'd like some info from anyone who tests it:
    - Map compile time WITHOUT Sentry
    - Map compile time WITH Sentry, as well as the number of worker machines you had.
    - Any anomalies in the final map's compile (Strange lighting, etc. Anything that looks different from a normal compile.

    Compile times:
    Hydro without Sentry: 9 minutes 58 seconds
    Hydro WITH Sentry, VRAD only, 2 workers: 7 minutes 4 seconds
    Hydro with sentry, VRAD + VVIS, 2 workers: 4 minutes 53 seconds.
    Post yours please :3
     
    • Thanks Thanks x 4
    Last edited: Jan 22, 2010
  2. The Political Gamer

    aa The Political Gamer

    Messages:
    4,468
    Positive Ratings:
    1,682
    <3 Fire you did it again.
     
  3. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    What? I was trying VMPI earlier this week and couldn't get the worker to be recognized.

    I thought VMPI for VRAD (the longest phase for me) was broken for the Orange Box engine part of the SDK. Did it get fixed, or is Sentry doing some sort of proxying-fix deal?
     
    Last edited: Jan 22, 2010
  4. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    Two things.

    One, you'll notice you need a special version of VRAD (It's in the instructions). Second, I spent the last few days tweaking around with the worker options (Many of them are hidden! Tricksy valve) until i got something that worked. It seems with OB, you need to run the workers in SDK mode, but force it to use the non-SDK packet distributor. :facepalm:
     
  5. Freyja

    aa Freyja ¯\_(ツ)_/¯

    Messages:
    2,834
    Positive Ratings:
    4,753
    <3 Now make a mac client so I can harness my mac and we're good to go?
     
  6. Lancey

    aa Lancey Currently On: ?????

    Messages:
    3,076
    Positive Ratings:
    1,314
    > What is a grue?
     
  7. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    This would require mac builds of vvis, vrad, and some other misc bits. Unlikely. :(

    You can try crossover though.

    The grue is a sinister, lurking presence in the dark places of the earth. Its favorite diet is adventurers, but its insatiable appetite is tempered by its fear of light. No grue has ever been seen by the light of day, and few have survived its fearsome jaws to tell the tale.
     
  8. Terr

    aa Terr Cranky Coder

    Messages:
    1,591
    Positive Ratings:
    405
    Ah, you built a new one from source. Verrahnice. So basically they forgot to change a switch in the file to use the "Works outside of our LAN" network setup?

    I'd suggest sending in the diffs/patchfiles/diagnosis to Valve... might make them fix the stock stuff faster.
     
    Last edited: Jan 23, 2010
  9. Cameron:D

    Cameron:D L6: Sharp Member

    Messages:
    363
    Positive Ratings:
    146
    Ok, I'm trying this out.

    I have 3 computers:
    lil-lappy
    server
    Main

    I'm compiling one of the Valve maps included with the SDK (sdk_pl_goldrush). When the VMPI window came up I added it to the queue, and now it seems nothing is happening :(

    EDIT: Its been like 20 mins, nothing. Cancelling.
    Will try again tomorrow.
     
    Last edited: Jan 23, 2010
  10. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    Hmm. that's strange. The setup looks good from here, and it still works for me.

    Grab me on steam some time today. We need to find out if my instructions are bad, or my code is bad.
     
  11. Nutomic

    Nutomic L11: Posh Member

    Messages:
    888
    Positive Ratings:
    177
    Submit Job and About give a 404 error, doesnt look too good.

    I dont have multiple pcs, so i cant test it, but i would love something that works online (no idea if its possible).
     
  12. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    Oh yeah dont' click the tabs. Site's totally not done. it's beta, remember?;)

    The submit job section is on the main page, just scroll down.

    Matt and I tested online compiling a lot. However, the SDK distributor is totally busted (Which is more optimized for higher latency internet compiles), and you'd need something like a 10mbit uplink to get much out of it.
     
  13. Cameron:D

    Cameron:D L6: Sharp Member

    Messages:
    363
    Positive Ratings:
    146
    Alright, I tried some compiling today...
    VVIS ran fine, but it gave a problem about having a different exe name for VRAD. I put the custom one into the orangebox/bin folder.
     
  14. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    Did you specify that EXE when you built the map? If building from hammer, it's tools -> options -> build programs, select vrad_LAA.exe. If you're not building with hammer, you'll have to figure out how to do this on your own; but most build environments should support changing the vrad exe.

    Or, if the error you got was a mismatch between vrad_LAA and vvis, then I need to tweak the delays a bit. But in the mean time you can simply re-submit the job.
     
  15. Ravidge

    aa Ravidge Grand Vizier

    Messages:
    1,544
    Positive Ratings:
    2,492
    Why does it need input on a webpage to work?

    Anyway, I have no idea how to set this up. Write a step by step instruction, or even better a video. Looking forward to it.
     
  16. Ezekel

    Ezekel L11: Posh Member

    Messages:
    818
    Positive Ratings:
    244
    nice, you've found a way to mimic the UDK's swarm system in source :)
    are subtractive brushes going to be next? (imagine being able to use subtractive brushes in hammer... could get some truly good timesaving here for complex shapes XD ).


    step by step guid like ravidge said would be nice.
     
  17. Cameron:D

    Cameron:D L6: Sharp Member

    Messages:
    363
    Positive Ratings:
    146
    1. Download the modified vrad (vrad_LAA.exe), put this in the %sourcesdk%\bin\orangebox\bin\ folder.

    2. Open hammer, go into the hammer options, to the build programs tab, and change RAD executable to
    Code:
    $SteamUserDir\sourcesdk\bin\orangebox\bin\vrad_LAA.exe
    3. Download the Sentry client on each computer you want to have compiling (Local network only), and run it. It should say something like "Sent connection string" in the console.

    4. Check the Sentry Website to see if your computers show up (in the workers section). If they don't show up, check your firewall etc (or the sentry server is bugged out, as it seems to be now)

    5. Start compiling your map (Not -final). After VBSP finishes, a VMPI
    tracker window should open, now go to the website and enter the details into the start job section.
    Enter the name of the map (Seems to work both with and without the .bsp extension)
    IP Address IP should be your external IP, default is correct.
    The Local IP should be set to the ip of the 'host' computer - the one with the VMPI tracker window open
    Leave port as-is.

    6. Sit and stare at the VMPI tracker window as all the pretty colours flash everywhere :>
     
    • Thanks Thanks x 2
  18. FireSlash

    FireSlash L2: Junior Member

    Messages:
    50
    Positive Ratings:
    49
    I restarted the server, is it still doing this?

    This is temporary. I'll write a script that can automate the process once i know it actually works. But I'm lacking (any) test data at the moment.
     
    Last edited: Feb 1, 2010
  19. chickenm4n

    chickenm4n L1: Registered

    Messages:
    49
    Positive Ratings:
    6
    quick! someone create a program that fools this program into thinking that the INTERNET is a lan connection!

    withe power of 4 billion computers at your compiling finger tips...
     
  20. Cameron:D

    Cameron:D L6: Sharp Member

    Messages:
    363
    Positive Ratings:
    146
    PROTIP: They need to be running the client :)
    I also had the thought of writing my own project for BOINC that will get a map and run the compile process on it, but it would probably take too long get the compiled map back to be worthwhile.