72hrJam Post-mortem Discussion

Discussion in 'Site Discussion' started by LeSwordfish, Aug 11, 2016.

  1. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,112
    Positive Ratings:
    6,065
    This is a thread for the after-the-fact discussion of the summer 72hrjam, and what we could do differently. I want to promote discussion here - so feel free to chip in with thoughts. I'm going to start off by going through some of the issues raised in response to the survey.

    • More or Bigger Streams - Good idea! I want to make the streaming and watching streams a bigger part of the Jam next time, but I'm not sure how - i'm not a big streaming person myself. Perhaps a round-up of good streams in the 24hr update posts?
    • Throwaway Entries and the Medal - I admit, I was dissapointed by the number of low-quality entries entered just for the medal. (Only about 1/10th of the survey responses said they only entered for the medal - but some of those would be lies, and the majority of people making low-entry submissions didn't stick around to do the survey.) Lots of people are calling for a minimum barrier for entry and i'm considering that... it's tricky to implement though. How do you distinguish a lazy entry from a simply poor-quality one? I don't want anyone being turned away because their honest effort isn't good enough.
      • Worth noting: we intended the medal to not be announced until afterward, but didn't communicate that to Valve. We were expecting the hype to have died down by winter - and it probably still will. We can prepare for it to be this big again, but I doubt it will be.
    • Finale Video - Again, good idea! We're looking at how we showcase entries now that there are so many (a dedicated site like the content pack sites is a possibility) and a video could promote some things excellently! Once again, all ideas are very welcome.
    • Easier Submission - This is something we were already thinking of. What I personally learned from it was just how few people read the rules, or the update posts, or the "How To Enter" post - maybe one in ten submissions was incorrect at first, and that's before the people who asked questions that had already been answered. I want to put together a one-button "ENTER HERE" link that sets everything up for you and makes it nice and easy.
    • More Tests - The testing we can do is, sadly, fairly limited: by the burnout on the players and hosts (see "Staff Involvement" below) and on the technical requirements (see "Technical Issues" below). Hosting additional servers was suggested - sadly, we rarely have enough players to run two successful tests at the same time, let alone three. We do have some concrete actions that we're taking on this, though: VBot has been improved to handle large numbers of maps better. The technical issues are being dealt with, and we're going to assign staff roles more strictly.
      • Some people also had trouble submitting to tests. I've put together a guide to this, which I'll be sharing around more obviously next time.
    • More Flexible Hours - Actually, i'm considering the opposite of this. Tracking the people who started early or late was a pain in the neck for me, and I'm considering just giving entrants a choice of two weekends, without allowing people to start a day or an hour early. Thoughts?
    • Technical Issues - Yeah, there were some pretty big issues. I personally take responsibility for the site downtime - I knew full well that the download system had filled up last time, and that this event was bigger - I should have known it would happen. That's nobody's fault but my own, and I apologise sincerely for it. Other issues - including the metadata loss - were completely outside our control, and have been patched so as not to happen again. Again, concrete actions:
      • We've patched the vulnerability that allowed the metadata link, and added a regular backup of that data.
      • We've figured what was filling the database so fast (a faulty, overzealous backup) and fixed it, opening about 22GB free space. What's more, we're going to do another purge and spring-clean before the next jam.
      • We're in the final stages of hiring two new members of technical staff - watch this space.
    • Full list of all entries - Perhaps. Hands up who would have read through a five-hundred-row GDoc sheet though?
    • WIP Thread - Good shout! I was expecting people to provide more help in the individual threads, but then there were so many of them! (I was expecting maybe 100-200 entries, tops.) I should definitely have been more pro-active in creating one once I realised the scale.
    • Make it a contest - No.
      • Ultimately, this event plays a different site role and acts in a different way to a contest. If you want a contest, it's not for you - you don't have to "waste your time" entering, as one response said. We do want to run events that cater more to the competitive mindset - we should be announcing the next one within hours - but this is no longer one. Sorry.
    • Staff Involvement - There's a couple of complicated issues here, and i'll try to go through them carefully because I know this is a serious complaint. Remember throughout though that first the sheer size and then the number of technical issues were unprecedented.
      • I'm bad at delegating. A lot of this is my fault - when we hit an issue, I tried to deal myself, and only passed it off to someone else if I actively couldn't. Again, this is the scale causing problems - by the time I realised it was too big for me, it was pretty late. For next time, I'm going to specifically grab a couple of staff members as dedicated assistants (they won't be entering themselves) and try to be more proactive in handing tasks to them.
      • As such, a lot of the reason other staff members did relatively little is because I didn't ask them to. There was never a time at which I requested help and was refused for anything but a good reason.
      • Yeah, it's a shame that Geit was given so much to do after leaving. This is because he dropped out quite suddenly, and it's taken us this long to get ready to replace him. (He did offer to keep helping out unofficially). Again, this is my fault - I should have been more proactive in bringing in new people.
      • I knew all staff members wouldn't be available to help sort through the results, so I asked for assistance from the veterans. When it came time to help sort them out, we had enough volunteers to make it an easily manageable task without bringing in too many staff members (several of whom now had jobs or lives to get back to.) As before, when I asked for assistance, it was immediately provided.
      • And finally I want to deal with a single, rather snide comment about staff who were "added recently without any clear reason." Listen, I know who you're talking about and, not to add a sour note, mate you need to do your fucking research. Killohurtz was by far the most helpful person throughout the jam, and has been proactive in community and site management since joining. I shan't hear a word against her - particularly from the same people that have been grouching ever since she was promoted.

    There were also lots of positive comments - I haven't gone through them in detail, but there was at least one kind comment or "nothing was wrong" for each suggestion or complaint. Thanks, folks <3 - as I said, the attitude and kindness of the community made this a pleasure to run, even through the stressful times.

    And as a final set of notes from the survey:
    • We aren't valve? We can't do anything about hitscan for you, friend.
    • Holy shit, don't put your steam password in the survey response, holy shit. It's people like you that are why we have the mobile authenticator, holy shit.
    • "give the community a a Unique cosmetic item or melee weapon like the saxxy for submitting something." I have some EXCELLENT news for you.
    • Someone is still blaming @Fr0Z3nR for causing the server issues. He's no longer staff... but if you have information, get in touch. Sabotage is not beyond him.
     
    • Thanks Thanks x 14
    • Like Like x 4
    • Agree Agree x 1
    • Funny Funny x 1
  2. MaccyF

    aa MaccyF Notoriously Unreliable

    Messages:
    908
    Positive Ratings:
    1,468
    hype
     
    • Agree Agree x 12
  3. Nicky

    aa Nicky Lets try something new!

    Messages:
    912
    Positive Ratings:
    1,698
    A good man is one who admits to their mistakes instead of making excuses and blaming others. Though I feel you put way more pressure on yourself in this post than you should, you are still a good man. In my opinion for the next jam, maybe everyone who participates gets a prize/medal thingy, but those you get in the showcase get a shiny/better one? Maybe similar to those unusual medals that staff got? Just an idea to throw around. I feel it will encourage people to not be lazy with some of their entries, and try their best with getting in the showcase to get a shiny cosmetic, but if they don't get in, they still get something, just a little less shiny.
     
    • Thanks Thanks x 1
  4. BenCo

    BenCo L5: Dapper Member

    Messages:
    242
    Positive Ratings:
    161
    Why not just give out the medal for those who got into the submission thread?
     
  5. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,112
    Positive Ratings:
    6,065
    The submission thread is a pretty unscientific thing - it was just me and Killo making the decisions, and "most worth highlighting" is not necessarily the same as "best". Tying rewards to the showcase would be a bit dodgy - people would have to be happy with possibly some arbitrary inclusions or exclusions, which i'm not sure they would be.
     
    • Agree Agree x 2
  6. Hi5TBone

    Hi5TBone L1: Registered

    Messages:
    37
    Positive Ratings:
    57
    Going to go on a little rant here.

    For the medal, I think it was a great idea. It gives the feeling that everyone wins in the end. But as you mentioned, there were way too many entries that were done up in less then 3 hours. I feel like it's unfair that mappers spend 45+ hours trying to get their map to even function properly, and people just come up with a guide in less then 1 hour and still get the medal. I think that there should be 2 different versions. One with the "TF2Maps.net 72hr TF2Jam Participant" and a "TF2Maps.net 72hr Mapping Participant". I'm not really good with names, so those probably aren't the best. Maybe I'm just being bias that I was one out of 200 people that made a map and think we deserve a special medal. Maybe I'm not, but you could tell that people just entered for it. Comments like "Do I get a medal?" "When's my medal coming in?" "I did it right, right??" For some reason, this really angered me a lot. I didn't think it'd be so hard for people to read the main post.


    All the posts asking for the medal made me angry that people kept asking, even after getting answered, and disappointed because people just entered for the medal.
     
    • Agree Agree x 2
  7. Hoplitejoe

    aa Hoplitejoe

    Messages:
    71
    Positive Ratings:
    146
    It could quite easily end up you get even more "quality entries" medal hunting once the item is out there in the game, getting noticed, especially from the people who play tf2 half as an item collection sim. I would at least just not mention the medal in any promo you do for it.
     
  8. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,112
    Positive Ratings:
    6,065
    Specifically with regards to this: I should have been clearer from the start when medals would arrive. It wasn't clear from your first quote that the medal wouldn't automatically arrive once it was correctly submitted.
     
  9. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,906
    • Agree Agree x 3
    • Funny Funny x 1
  10. Joshenkstone

    Joshenkstone L5: Dapper Member

    Messages:
    207
    Positive Ratings:
    177
    It would be cool if there was an official tf2m stream on twitch.
     
  11. Zurby

    Zurby L2: Junior Member

    Messages:
    76
    Positive Ratings:
    109
    In the survey I chose that I only joined in for the medal. Not because I'm a dirty leech.. I think.
    I found my way to this place by going through twitch tf2 and looking for level design or art streams.

    Though when I got here the jam was 8 hours away from finishing. The free medal was a great motivation to learn everything I can about making tf2 models, making one and getting it done in time.

    My entry was not very great and it was just a hat, so definitely not a lot going for it. So I'm still very thankful for the medal. :)

    (By the time I finished getting all the programs, all the files, and the model and texture there was only like 20 minutes left of the jam, close call I say.)
     
    • Like Like x 1
  12. EArkham

    aa EArkham Necromancer

    Messages:
    1,576
    Positive Ratings:
    2,199
    I don't understand why the medal is such a motivator. It's untradable, and let's face it, chances are you're going to prefer using that cosmetic slot to wear something more visible. I'd think it has more value to the regular mappers here as a (literal) badge of honour than it would to randoms just looking to get another item. And I say that as a big time collector of TF2 items.

    The suggestion to have two -- one for participating and one for actual mapping, I guess? -- doesn't seem like a bad idea, but breaking it into multiple medals also means judging. If only "lite judging." I don't have a problem with that myself, but my impression is that judging was the big thing to avoid with the 72 hour events.
     
    • Agree Agree x 3
  13. HolySnickerPuffs

    aa HolySnickerPuffs Bad at TF2, Loves Mapping For TF2

    Messages:
    931
    Positive Ratings:
    1,255
    I just want proof that I've done something on TF2Maps.net.
     
  14. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,072
    Positive Ratings:
    834
    Great post, Sword.
     
    • Agree Agree x 2
    • Thanks Thanks x 1
  15. Grizzly Berry

    aa Grizzly Berry

    Messages:
    431
    Positive Ratings:
    341
    I think this is something our community should focus on. Twitch is a great platform to advertise TF2Maps, especially during the 72hr. More streams is great, but I don't want people getting discouraged because they aren't getting viewers. Let's face it, unless you're a prominent mapper (and even then), it's hard to get viewers and keep them. I (or @Crash) could probably whip up a guide to streaming from an entertainment standpoint if anyone is interested.
    Not a whole lot we can do about this. A lot of people want their map tested, but don't want to playtest themselves. It's unlikely the public will want to test maps, and if they do, who's to say they'll provide valuable feedback. Unless you have a dedicated audience like Crash, I think getting testing time is going to continue to be difficult.
    I agree. It was acceptable when there were only a few people needing to work outside the scheduled weekend, but with so many entries, it's just not possible to keep track of them all. The 72hr is ran twice a year. Anyone who misses the first, will have another shot later.
     
    • Agree Agree x 2
    Last edited: Aug 12, 2016
  16. Hotel Detective

    Hotel Detective L4: Comfortable Member

    Messages:
    187
    Positive Ratings:
    189
    People are crazy about cosmetics man, I spent a year of my life just to get a Wiki Cap. I didn't do it FOR the wiki cap
    (I stayed at the wiki cause I liked the community at the time) but it was definitely an incentive that stuck in the back of my mind when I made edits.

    Same thing with the medal, I normally don't release anything I make in Hammer, but the medal finally took me out of that shell
    (I've been using Hammer for TF2 since 2009, this 72hr jam was the first time I released anything publicly)

    Idk. Having a reward makes people feel.. well rewarded.
    Plus I mean fucking hell it's JIMIJAM ON YOUR CHEST WHO DONT WANT THAT AMIRITE
     
    • Thanks Thanks x 1
  17. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,112
    Positive Ratings:
    6,065
    I'd love to see something like this - particularly in time for the 7.2 hour contest!
     
    • Agree Agree x 1
  18. Grizzly Berry

    aa Grizzly Berry

    Messages:
    431
    Positive Ratings:
    341
    Done!

    Of course, if anyone has any questions, they are more than welcome to ask them in that thread. We tried to cover most of the basics.
     
    • Thanks Thanks x 1