72hr Contest Improvements

Discussion in 'Site Discussion' started by Turbo Lover, Aug 5, 2015.

  1. Turbo Lover

    aa Turbo Lover Fight me under Glasgow Central Station

    Messages:
    333
    Positive Ratings:
    317
    Chat's been buzzing with talk about the 72hr and some people think how we do it sucks, and maybe it does suck, post your hot opinions in here.

    Some things to consider:

    • Judging
    We have close to 70 entries in this most recent contest, that's a lot! Nobody likes testing that many maps, and although we have a preliminary round of voting set up to help whittle down the entries, we're still going to be testing the same versions of the same maps for 2 months or so. Do you think the process of thinning out entries should be more rigorous? Should we allow authors to release new versions of 72hr maps while their entry is being considered for the contest?
    • Prizes
    Some people in chat have also brought up the idea that this contest shouldn't be a contest at all, and should be more of a personal challenge for the authors, this would entirely eliminate winners, prizes, and the voting process. Do you think the 72hr needs a competitive aspect as motivation? Do you think if we removed the competitive aspect that it would affect the number of participants?

    Let us know what you think, and if you can think of anything else that would improve future 72hr contests, let us know that too. We can't change the process of the contest that's already running, but we can probably do better next time.
     
    • Thanks Thanks x 2
  2. Kwzby

    Kwzby L1: Registered

    Messages:
    21
    Positive Ratings:
    5
    You could split the maps by game modes test each group individual with judges for those specific catergories.
     
  3. Jethro

    Jethro MUSty Complainer

    Messages:
    288
    Positive Ratings:
    259
    RE: Judging/thinning out... honestly with the SHEER amount of maps that are a part of the most recent contest, I think thinning out HAS to be more rigorous. As much as that is unfair to maps/mapmakers, the simple fact is sixty-six maps takes a LONG time to test. As it is there's a schedule, and there's something like 12 maps per week being tested, or something. That's going to be around 4-5 weeks to get all maps properly tested.

    RE: Prizes I sort of doubt anyone doing this is in it for the prizes. I mean sure, there's the prestige of getting first place, and all that, but I'd like to think everyone is in it for the pure challenge of it. Of course I like to think the best of peoplepahahaha i can't keep a straight face. But yeah, I could go either way regarding prizes/winners/voting.
     
  4. Idolon

    aa Idolon the worst admin

    Messages:
    1,536
    Positive Ratings:
    4,404
    I was honestly expecting a lot more people to drop out of the judging phase (nobody has taken me up on the offer yet!). Granted, I'm still in this thing, and I can't even win any of the prizes. I guess knowing where your map stands in comparison to others is a big want!

    Getting through the preliminary phase faster would be a very good thing, though I'm worried that we'd be giving little or no feedback to the maps that need it the most. That said, the most broken maps are probably the ones that have the most blanket/obvious feedback (overscaled, underscaled, dark, maze-like, etc), so maybe it wouldn't matter too much? I'm not sure.

    Releasing new versions of maps while they're still in the running is a definite no.

    Getting fixed versions of maps after the deadline of the contest is a real pain in the ass for a myriad of reasons, so maybe just disqualifying broken maps instead of allowing a fix would help cull the herd a bit.
     
    • Thanks Thanks x 7
  5. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,742
    70 entries is a great number!

    But...

    As I've said before I believe this should be a personal challenge event rather than a contest.
    That means no judging, no prizes, just hearty pats on the back all round.

    Turbo Lover suggested in chat that this would make the whole venture worthless.

    But I disagree.
    The value of the 72 hour contest comes from the personal challenge, the fact that everyone is working furiously at the same time, the collective streaming, the lost sleep, the fact that no matter what you made, you're probably damn proud of yourself.
    The value of the contest doesn't come from the prizes, or from winning. Finishing something you're proud of in 72 hours is winning.

    The contest as it is, is fine.

    The judging and voting, I think should be entirely scrapped because it's awful. It's a god damned nightmare.

    Here's why:

    We've got 70 maps to play and judge, that means it'll be like two full months before this process is over. No one can update their map in that time unless they pull out. (as stated in chat, anyone is welcome to, but no one will, even if they know their map is broken, they're not going to pull out, it's never happened before, it won't happen this time).

    This, on the surface, seems like a good thing, but it's brutal. There are three categories of maps I'll refer to:

    The broken maps <- these have horrendous lighting problems or entity issues. All of these problems could be fixed with max 20 minutes of tinkering.
    The bad maps <- these have major issues because the author isn't experienced yet or they just didn't get it right this time. It happens, no one can be expected to make something good every time. These ones would take a few hours of reworking to make something that might be OK or good.
    The rest <- these maps are either tolerable or actually pretty good.

    Here's how two months of judging hurts these three categories of maps:

    The broken maps - They could fix their biggest issues in 20 minutes and a recompile. But they're not allowed to. So they get played over and over until everyone who wants to vote has played them, everyone has seen the same set of problems. The author is inundated with essentially useless feedback "Fix ur lites nub" They could have fixed the problem but no, instead the entire testing time for them has been worthless.
    2 months of worthless testing

    The bad maps - These ones could have seen the major flaws after just one playtest and have got to work fixing them. But instead they get the same feedback over and over until everyone has played them. "A is impossible to attack" etc. The first test was useful for them, everything afterwards is a waste.
    1-2 good tests, 2 months of worthless testing.

    The rest - These ones get some great feedback, people comment on them as being good so they're fresh in peoples' minds when the nomination and server voting comes on. They get played lots and each time the differing team makeup and players reveal different things to provide feedback on. They could have put out a new version to get some new feedback, but they're not allowed and they know their map is all right, so definitely don't want to pull out. If they'd miss out on all the rest of the feedback and wont get to see how their map ranks against all the others.
    They get great feedback but returns diminish and they get nothing new after the first month.

    But wait! There's another category of maps affected by this judging;

    The non-72hour maps - These poor maps are sidelined for the first few weeks, there are 70 maps to play, so there's absolutely no time for anything else. After the first two weeks they might be able to squeeze into a spot or get lucky and see the gameday thread before it's dogpiled by everyone else with a non-72hr map.
    They're flat out told no in the first two weeks, afterwards it becomes decreasingly difficult to get testing.

    As players how do these maps affect us?

    The broken maps - *sigh* we're playing this again? I guess half the people here haven't yet so I'll just go through the motions until they see how broken it is and we can RTV and move on.
    Bad experience repeatedly for 2 months.

    The bad maps - Hooray it's not broken! Let's try and play extra well so that we can push through that impossible choke! It's still not very fun though and generally a bad time, but other people on the server might enjoy it and need to know what to say for feedback so we can't RTV it too quickly.
    Meh experience repeatedly for 2 months

    The rest - Yeah OK, maybe I actually enjoy this map. I can have fun playing this.
    Good times for 2 months

    The non-72hour maps - Why are we playing this? There are entries to judge RTV
    2 months of RTV


    This is why I feel the voting is a bad time for everyone involved, beyond the first test or two it stifles the quality of feedback someone can get and it can get really frustrating for people doing the testing because of the sheer volume of bad experiences that they need to evaluate.

    I really, strongly, feel like we should ditch the entire process.
    Instead we should do 1-2 weeks of highly organised testing. Gameday style events. Make sure that everyone's map is run during an event at least twice. All the demos are posted for people in one centralised thread. Beyond that, it's up to regular imps and gamedays to get testing for your 72hr map.

    This way authors would get their testing. So the new members can have the awesome experience of getting their new map tested. But after the scheduled events, people can continue to make changes to their maps as normal. Nobody's flow would be broken, if people are pumped up on mapping they can continue and make as many changes as they like as early as they like and I think people would, I think more of the 72hr maps would end up as finished maps that way. By the end of the voting period people are fatigued of their own maps. And that's bad.


    Appendix A:
    Tyler if you say I'm overreacting or getting emotional I swear to god I will overreact.

    Appendix B:
    I said all this last contest and the contest before, Last time there were 40ish maps. It's the single largest reason I didn't finish mountainside and sabotage, I felt they were good (and the vote results show that) so I didn't want to pull out to continue working on them despite the fact I was pumped up and wanted to. By the end of the two month process I was fatigued or doing something else. So they fell by the wayside.

    Appendix C:
    If you read this entire post, well done. You deserve a prize!
     
    • Thanks Thanks x 21
  6. Jethro

    Jethro MUSty Complainer

    Messages:
    288
    Positive Ratings:
    259
    Honestly, most of your points (other maps getting shafted, being essentially "forced" to test an old version of a map that the creator already knows is all kinds of wack etc) are why I lean toward doing away with the whole contest thing.

    Or atleast, toning down the whole contest thing so it's much less of a "big deal" if your maps get tested or updated. Maybe have it as a thing that some (not all) maps are singled out and put into the running to win, but you'd be allowed to update it and stuff from then. I don't know, it is mainly about doing a map in seventy two hours, but I don't think it should then continue to be testing that same map for two hundred plus hours.
     
    Last edited: Aug 5, 2015
  7. MaccyF

    aa MaccyF Notoriously Unreliable

    Messages:
    907
    Positive Ratings:
    1,467
    ---Prepare for a ramble---

    I think broken maps shouldn't be updated, but should be allowed to roll back to their previous version in the spirit of fairness.

    I also think a theme for each 72 hour would be a good idea, it makes it easier to quickly make decisions on which do the theme best. The theme could be aesthetic or gameplay related (e.g., dynamic payload)

    I have to say though, having been in the imp/testing earlier, it did seem that a lot of were pretty quickly disregarded by the players, for mostly fairly obvious reasons (like almost non-existent lighting or massive overscaling)

    Perhaps we could have a team of volunteers to whittle down the entries before going to general testing.

    As regards the prizes, in some ways i agree that they aren't necessary, as most people who compete probably do it for a personal challenge (like YM thinks) but on the other hand i'm sure they encourage new mappers to have a go with the SDK.

    I think that's about all, if i have any other thoughts i'll add them.

    ---Ramble Over---
     
  8. tyler

    aa tyler snail prince, master of a ruined tower

    Messages:
    5,033
    Positive Ratings:
    3,980
    I think you make good points YM, it just behooves one to arrange them neatly (as you've done) rather than rant over several lines.
     
  9. Vincent

    aa Vincent 🔨 Grandmaster Lizard Wizard Jedi 🔨

    Messages:
    914
    Positive Ratings:
    580
    I considered withdrawing Sakura Hill because I really want to fix it but I want to see how I stack up to everyone. Prizes be damned, I just want a map maker's medallion (which is only first place, that's not happening for me).

    That goes against the spirit of the competition, it's all about what you think you can do in 3 days.
     
    • Thanks Thanks x 1
  10. MaccyF

    aa MaccyF Notoriously Unreliable

    Messages:
    907
    Positive Ratings:
    1,467
    It goes against the spirit of the current format a bit for sure, that was just a thought about making it easier to judge
     
  11. Fantasma

    aa Fantasma

    Messages:
    886
    Positive Ratings:
    980
    This should be the map author submitting properly, last time I uploaded regularly, and kept a link to the older version in the post in case the latest one was bad

    This always got criticized, I dunno if it just the ones I saw, but a lot of people like myself felt too constricted to make something they wanted, which is essential in 72hr
     
  12. shadowslasher11

    aa shadowslasher11

    Messages:
    392
    Positive Ratings:
    325
    Alright, then what if we gave say...3 themes a contest. For example, Alien, Egypt, and Urban.

    That way we can at least give a bit more freedom for map makers do do what they want between the three themes. I feel like if you allow total freedom of making maps in these 72hr contests it brings the situation to the table that we will end up having too many maps at some point.

    This could limit it in a way.
     
  13. RubbishyUser

    RubbishyUser L7: Fancy Member

    Messages:
    414
    Positive Ratings:
    242
    I know I put my "thanks" on there but I want to reiterate what YM said. I made a 72 hour map, but I actually gave up and didn't even submit an a1 because it was massively underscaled (I'm rusty) and I didn't want to waste any more time. It's bad enough fitting all this work into 72 hours and then the fact is that the time afterwards judging is AN ORDEAL. I stopped playing TF2 altogether after the last one. Is there even a map from the last contest that's still in active development?

    Maybe without the 72hour "contests", we can have more proper contests, which mean a much less intense testing period and fewer entrances. The 72hour periods can then be more like "open days" - time when people who wouldn't normally map are encouraged to get involved and we all get a gold star for participating at the end.



    Having said that, we need to consider those who mapped for the first time with this 72 hour contest. Would you have mapped without a promise of a prize at the end? What are your thoughts on this experience in general? Is your map one of the "broken" or "bad" ones that is mentioned above?
     
  14. biskuu

    biskuu L2: Junior Member

    Messages:
    97
    Positive Ratings:
    17
    This was my first contest and although my map wasn't very balanced I managed to detail and optimize it a lot and it took me like 35 hours or something. I think if you have a basic map layout in your mind before the contest, you can easily have a ready to be detailed map in less than 25 hours.
     
  15. Grizzly Berry

    aa Grizzly Berry

    Messages:
    431
    Positive Ratings:
    341
    I like the prizes. I don't think the majority of mappers are doing it to win, but it's a nice little bonus for those who did a great job at putting together a map in 72 hours.

    I do agree the judging process is tedious though and I really dislike having to play the same maps over and over. I'm not sure if there's a better way to judge or not though.

    I guess what I'm trying to say is I like having a top 3, but I would still enter without the chance of winning anything.
     
  16. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,109
    Positive Ratings:
    6,058
    Yeah - the contest aspect of it is nice, but at the moment is looking like a detriment to normal activity.

    So perhaps the answer is to strip down the "contest" aspect a lot. Maybe a early stage of judging could be to weed out YM's first two types of map with a quick judging panel, but i really dislike how the first part of judging will be to just tell a bunch of people "no, we dont like your map." and nothing else. That's not a good way to get value to the people who need it most.

    Ooh, perhaps we could split it. Everyone nominates a category they want to be included in: new mappers or veterans, and the judges have some capacity to move people between it. We can then focus our attentions in different ways where most needed: feedback for the newer mappers, judging for the veterans.

    I kind of like the idea of the guaranteed testing. What if we were to remove the prizes and have an informal vote at the end of testing where people choose a few favourites? One thing we had before was a google form for people to say "yes", "no", or "have not played" - that's good, perhaps that could be the whole process? You dont have to play all maps to give opinions on some.

    I'd like to add that just because we're only complaining discussing this now, does not reflect badly at all on Idolon's handling of the contest - he's done a great job and worked hard for this, and any issues are either inherited or due to the unprecedented scale.
     
    • Thanks Thanks x 8
    Last edited: Aug 6, 2015
  17. Vincent

    aa Vincent 🔨 Grandmaster Lizard Wizard Jedi 🔨

    Messages:
    914
    Positive Ratings:
    580
    I don't think anyone expected the volume of entries we got this time around. I was incredibly surprised myself. 60+ is pretty crazy.
     
  18. CommodoreKong

    aa CommodoreKong

    Messages:
    332
    Positive Ratings:
    303
    Has the 72 hour map contest ever had so many entries before? I seem to remember having maybe 30 or so entries last time. I'm guessing that this contest just happened to hit at the right time, with the Gun Mettle update so recent people know that Valve is going to be a lot more interested in custom maps in the future vs the last few years of TF2 and maybe more people then normal decided to enter the contest because of Gun Mettle. It's possible future contests wont have nearly as many entries.
     
  19. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,742
    I did. We had 43(iirc?) last time and there have been a lot of new members recently.
    Valve put up the workshop, Crash started his tutorial series, natural site growth, they all combined and sometime mid June I said we'd probably hit 50+ easily and probably more like 60+

    It was a pain to get all the maps done last time when there were 40, now we have 70, and I think we're either going to get more next time or at very least similar numbers.

    RE: Ido's handling of the contest: absolutely fine, these issues existed before and were debatably unimportant, with the growth though, they need to be addressed.
     
  20. RaVaGe

    aa RaVaGe

    Messages:
    728
    Positive Ratings:
    1,053
    About the judging, why not putting a rating plugin in the server ? Something from 0 to 20, like that you are less likely to get equalities in comparison of a 0 to 10, then you get a natural trend about the maps that are good or not, then takes the top 20 maps.

    You can't expect people to test 70 maps, to rate them, and then to leave feedback, it's not possible, you can ask that for 20 maps not more.