The 'A Tale of Two Skill Sets' Contest Post Mortem

[Fruity Snacks]

tl;dr: it was the best of times, it was the worst of times.

So this has taken a little bit longer than I had hoped to write, but here it is now! A post-mortem of the 2SS contest. These are my personal views on the contest from the standpoint of a community manager based on observation and conversations that I've had with you guys, privately or in steam chat. They're a little scattered because I kind of rushed this and am pretty scatter brained at the moment, but here we go.

(As a disclaimer, this views are in no way representative of the other staff members, or of the community as a whole. These are my personal feedback points on the contest)

The format:
The format for the 2SS contest was different that what we have had in previous contest. It was basically 2 minor contests combined into 1 major contest. Initially, this seems like an okay thing: people could enter into whatever phase they wanted and play up their strengths. However, after chatting with various community members (and through my own observation) this format was actually more harmful to the development process, than it was helpful. By dividing the contest into a layout only phase (no detailing) and a detailing only phase (not layout changes, and actually punishing people who did), it effectively was dictating how people should be developing their map. This breaks people out of their normal, comfortable routines for map development or restricts them from developing their own process if they are a newer level designer. This is an added stress and limitation that can make things hard, even for an experienced designer. While limitations from contests can make things fun and unique, the limitations shouldn't be on the creative process itself.

The other issue that popped up was mapper burn out. In every contest there is the 1 week of pure hysterical mapping/polishing before final submission. If you've been there (and many of us have multiple times), it can really take a lot out of you, sapping your mapping juice for a few weeks. With the format of the 2SS contest, there was going to be two of these crazy mapping weeks for the entrants who entered both phases. Before someone could even catch their breath after the first stage, they have to jump right into the second stage. Tie this in with dictated development process, and it becomes a pretty uncomfortable contest with a lot of un-necessary stress.

Deadlines:
When setting deadlines for any contests, you want to make sure you don't put it smack dab in the middle of someone's finals week. With the Mercs Vs. Aliens contest being planned at the same time as the 2SS contest, and knowning that MvA would be after 2SS, I wanted to give plenty of time for people to decompress before the MvA contest. You want to give mappers enough time that they can finish their entry, but not so much time that they forget, or get bored. We used the original Valve Artpass Contest as a metric for the Artpass phase.

That all being said, if you see a deadline that lies in the middle/near a big national 'thing' (like finals week for university in Europe) say so! It's better to say things earlier, rather than later. The closer the end of the contest is, the less likely the host could be to change the deadline. Conversely, if you see a deadline that is going to conflict with something, plan ahead and don't put things off until the last week!

The community:
The turn out of the first phase was just about as expected: lots of people. The surprise, however, came in during the artpass phase. Barring the awkward deadline, there was not many entrants into the artpass phases, even after the innumerous requests to have an artpass contest. I quite honestly expected to see a lot more artpass entries (or potential entries) than I did. I haven't been able to figure out entirely why this might have occurred. My initial observations suggest that it was because of the time limit of the second phase being too short. I also thought that it might have something to do with the previously mentioned reason of dictated development process. Perhaps it was just prizes? I'm not 100% sure, but if it came down to “people just don't like to do detailing” I wouldn't be surprised. If you guys have opinions on that, throw 'em out there.

Conclusion:
The 2SS contest was a nice 1-off contest. I would consider it a fun, successful contest, but only as a one-time thing. Its format (or anything similar to it) is not something I would suggest or recommend in future contests. The seperated deadlines lead to a unique event, but made the overall event feel too forced onto the entrants. As I mentioned before, limitations in contests are fine, but not if they limit the creative process of the individual entrants.

 

[fubarFX]

I guess I'll pop in and say a few words here (not trying to be pertinent or anything here) just to get the discussion going.

I had sworn to never enter a contest again but did it anyways because the minimum commitment was so low. I was given the opportunity to just wip out a generic map with no theme in mind and have a little bit of fun with gameplay and be done with it. Which is why I have only entered in phase 1.

I did tug-o-war pl, for the heck of it, it's a fun mode to explore, altho it's greatly underrepresented and underappreciated. having a tug-o-war map in there for the sake of variety just felt like the right thing to do even tho I knew for a fact that this would probably not be something anyone would have any interests in. but I did it because I felt like it. It was what I wanted to do and had fun doing it without having to overcommit and that was that. not really sure how the map was received nor did I care. I had done a great deal to polish a gamemode that never really worked and that was good enough for me.

my map now populates the orphaned subforum as it was planned to do and I feel okay about it. I made a pretty disinterested map for a contest I had pretty much no interests in, and somehow that worked out great for me. Of all the contests I have been in (comp ctf, artpass, sym/asym, koth and the earlier 72hs) this has been my most positive contest experience so far and I'm happy I was a part of it.

 

[Fantaboi]

Being able to edit the phase 1 maps in the MANN V.S Aliens contest would be cool

Edit: Aliens, I meant aliens.

 

[Berry]

I personally enjoyed the contest quite a nice bit, and had fun entering. I ]would like to see more experimental contests mixed in with the typical 'normal format + make it for this mode or make it for this theme or subtheme'.

However it could have been formatted quite differently. I don't have any effective personal suggestions, but I would like to see more in the future.

 

[YM]

my tl;dr is couldn't enter; was busy.

If I hadn't been busy I may well not have entered the latter part.
The first part seemed good, nice time limit, my 72hr method meant I had something that wasn't suitable to reuse for the main contest, I was open to the idea of doing something separate for it anyway but between wanting to continue work on the 72hr work and knowing I'd be busy later in the contest meant I didn't invest any time into it.

Then for the second half it fell apart for me. Being locked down to a layout was a killer because you can't react to any problems that only become noticable later, if you're arting it up and discover one of those suddenly you're at a wall and that'd really destroy my motivation. At the end of the 1st stage I'd feel i'd have to artpass my own entry, but I wouldn't really want anyone else doing the same if I was. Just feels weird to me after the Valve artpass contest that two people would indipendantly artpass the same map. The time limit also felt a little short.

 

[UKCS-Alias]

To me the contest itself wasnt too bad in the idea (read: i loved it) and i still believe a 2nd contest is actualy possible by making those restrictions less harmfull yet still exist.

However, i still think the contest would need some more drastic changes to work.

First the deadlines part. Especialy in phase 2 you want to be sure you are detailing a quality map. However, you realy couldnt know which of the 20 entries would be good. So people only focussed on a few maps and lots of others got ignored. I rather would see the 2 phases split until phase 1 has its vote results. This might remove alot of maps that are going to be used in phase 2 (as for example 10 out of 20 entries will show poor results compared to the others) but will still add a few (instead of 4 maps you can for example have 8 which all had a 7+).

2nd, the 2skillset idea was that 2 mappers would be making 1 map. The contest however didnt realy show that as keikoku was the winner both times by aly. It was still more rewarding to keep to your own map since you know it best and what its detail target was supposed to be.
The next time i rather see the person that created the map in phase 1 excluded for phase 2 on that same map, he must detail someone elses map. And this means that you can also allow more detail in the alpha phase. The other mapper can easily decide to replace it.
As side effect you also will solve the issue of having a slightly overdetailed alpha map in the first phase. You cannot use that overdetailing on the 2nd phase since you cant work on your map. Alternatively you could still allow them to finish it and allow them in the voting stage, but that would need a penalty on the score (for example exclusion or a -1, which is harsh enough to be able to exclude you from the winners unless it was excelent)

Being able to edit the phase 1 maps in the MANN V.S Aliens contest would be cool

And this is something that also could work. Allow the undetailed maps to get a 2nd chance to still get detailed by someone else. The maps were designed for it after all.
Still, it would give a headstart on the maps as for example a rating on gameplay already is known from a previous contest (and they cant drasticly change or there is somethign wrong with the voting process itself).

I would recommend a 2nd detail contest of the phase 2 maps, just for the maps that werent detailed (or for example ended up as a poor map in the voting stage, in that case a new attempt might not be bad).
Personaly i wouldnt even complain about it and instantly submit intercept for that again (after all, im still waiting for that moonbase pack with items and textures before i even start detailing it).

Then for the second half it fell apart for me. Being locked down to a layout was a killer because you can't react to any problems that only become noticable later, if you're arting it up and discover one of those suddenly you're at a wall and that'd really destroy my motivation.

And this is also a good point. If a map shows a major flaw for your details to actualy work, or it has a part in gameplay that is broken people should be allowed to fix it.

There are many maps that still get gameplay changes in phase 2. This is as YM already mentioned an unneeded restriction and it does need an exception rule on it. If there seems to be a place in an MvM map where bots get stuck then the mapper must be allowed to change it. Or if gameplay is harmed by having a major overpowered sentry spot then the mapper must be allowed to cover it up for balance reasons.

Only on major changes that massively affect the gameplay by making the map behave diffirent shouldnt be allowed. A new tunnel is an example of a major change. Splitting a path by having a rock in the middle might not heavily affect gameplay - or it might be the fix for that sentry - and for that reason could be allowed.

 

[LeSwordfish]

I just want to quickly talk about some of the logic behind this contest- I've discussed it in chat, but never on the forums.

An artpass contest was an idea we were throwing around a lot before this contest started. I won't go into too much detail here, since we haven't completely discarded all the ideas we were fiddling with, but we were thinking about playing with the artpass format, but sticking to a particular map.

I didn't like this. My problem with an artpass contest was, only one person gets anything out of it. Everyone makes their own version of, say, cp_desert, and at the end of it, two or three people "win". They get prizes. Yay for them. But what happens next? Only one cp_desert goes into rotation on servers. Only one gets popular, and only one gets its maker any notoriety. Everyone else goes home empty-handed- completely so. In a normal contest, at the end of it, everyone has a map that they can finish, change, be proud of, and pimp to server operators or to valve or whoever. In an artpass... you don't. Why would we want cp_desert? We've already got one, and it looks better.

(People were willing to join the valve artpass contest because the rewards were far greater, I think. Same risk, far greater possible reward. Also: sure, you get a portfolio piece. But you'd get a portfolio piece spending the same amount of time on your own map, too.)

So I wanted an artpass contest that fixed that issue. My first thought was an "orphaned maps artpass" where you pick a map from either a pre-chosen list, or just any orphaned map, and detail it. That way, you have a cool unique useful map, and you can win a contest.

Another problem with artpasses: if you find layout more fun, you're left out in the cold. This was where the two-stages idea first came in- first stage was to build a map, then we chose a winner and everyone detailed it. It was pretty natural to blend the two ideas together for what we have now. Other suggestions that were floated included allowing people to detail only the top three/top five layouts, and allowing anyone to detail any entered layout, but only after the layout judging phase. We were already planning on having the two stages concurrently, so as not to lose momentum, but the strict timescale of the MVA contest made the decision for us.

---

Another thing for discussion: Collaboration in contests. I was against it for this contest, mainly because people tend to collaborate only with people they know are good, which a) shuts out newer mappers and leads to more of a "boys club" feel to the site, and b) ends up multiplying talent, meaning that the contest is almost decided in the collaboration choices phase. Oh, YM and Rexy are collaborating? Grazr's doing the layout and Aly's doing the detailing? Turbo's doing the normal popfiles and alias is doing the Expert ones? Everyone else go home.

Another idea that's been floated is formalising collaboration as part of the contest design- for example, randomly selecting collaboration partners. Of course, there's various reasons why this wouldn't work. (Oh, i got paired with marioboil33ttrademapper2003? Never mind then.) But they're interesting ideas to throw around, too. What do people think?

 

[Fruity Snacks]

I'll go over a few of the points you made, and give my thoughts on them:

First the deadlines part. Especialy in phase 2 you want to be sure you are detailing a quality map. However, you realy couldnt know which of the 20 entries would be good. So people only focussed on a few maps and lots of others got ignored. I rather would see the 2 phases split until phase 1 has its vote results. This might remove alot of maps that are going to be used in phase 2 (as for example 10 out of 20 entries will show poor results compared to the others) but will still add a few (instead of 4 maps you can for example have 8 which all had a 7+).

We thought about that originally, but felt that having a vote, then starting phase two would be worse than what we had. We wanted to give all maps a fair chance at getting detailed, and wanted to remove any biases that might pop up from voting before starting the detailing stage. Also, the voting phase would've made the contest really feel like 2 smaller contests, not one cohesive contest that we wanted.

2nd, the 2skillset idea was that 2 mappers would be making 1 map. The contest however didnt realy show that as keikoku was the winner both times by aly. It was still more rewarding to keep to your own map since you know it best and what its detail target was supposed to be.

This wasn't the intention. It was our way of putting in collaborations, without saying "collaborate!" I it was a 2 mappers making 1 map thing, we wouldn't noted it more in the rules. Basically, we wanted to give mappers/artists a chance to enter a major contest, but only do the part they felt they were strong at, or wanted to get better at, not necessarily "2 mappers make 1 map."

The next time i rather see the person that created the map in phase 1 excluded for phase 2 on that same map, he must detail someone elses map. And this means that you can also allow more detail in the alpha phase. The other mapper can easily decide to replace it.

This is just a really really bad idea, in my opinion (sorry for being blunt). I would NEVER want to tell someone to work for a few months on a layout, then not be able to continue with it through artpass. That just seems really rude and insulting towards the layout designer.

As side effect you also will solve the issue of having a slightly overdetailed alpha map in the first phase. You cannot use that overdetailing on the 2nd phase since you cant work on your map. Alternatively you could still allow them to finish it and allow them in the voting stage, but that would need a penalty on the score (for example exclusion or a -1, which is harsh enough to be able to exclude you from the winners unless it was excelent)

Again, it is ALWAYS a bad idea to punish someone for working on their own work. It's a great way to get people to not enter the contest (again, sorry for being blunt).

And this is also a good point. If a map shows a major flaw for your details to actualy work, or it has a part in gameplay that is broken people should be allowed to fix it.

Fair point, but this is something that the designer needs to keep in mind, if they don't expect to detail the map themselves. Conversely, it's also something that the artist should learn how to tackle: getting creative within some set limits.

There are many maps that still get gameplay changes in phase 2. This is as YM already mentioned an unneeded restriction and it does need an exception rule on it. If there seems to be a place in an MvM map where bots get stuck then the mapper must be allowed to change it. Or if gameplay is harmed by having a major overpowered sentry spot then the mapper must be allowed to cover it up for balance reasons.

The rule to not change the layout was to protect the layout designers original vision they had for the gameplay of the map. Being able to change the layout after the layout stage is done, would then be kind of moot in a 2 skill set type contest. It might as well be just a normal major contest without the focus on the layout phases and the artpass contest.

EDIT: Ninja'd by flippin Swordfish.

 

[LeSwordfish]

Ninjas are usually stealthy, this is just one word-bomb hitting a little before the other.

EDIT: I'd like to point out that "not punishing someone for working on their own map" is a far better phrasing of my long-winded second paragraph.

 

[fauks]

It seems fair to me that winning one phase excludes you from winning the other.

 

[Sergis]

the first part turned out alright because spamming alphas is what we know and love and are very familiar with
the second part had fuck all entries because we have fuck all people who actually get to detailing and have enough experience with it to be done on time

basically "here's two months to do what you do best, now here's two months to do what you've never done"

sure people were asking for artpass but people tend to overrate themselves. people were excited about batt contest - only 5 people actually managed to build something. dynamic payload was to be all creative and shit - most of the maps did either an elevator or an animated cart delay.

sure we have individuals who can handle random challenges but mostly tf2m is comprised of people who still need to fully explore the box called "alphas" that contests want them to think outside of. gotta take that into account when designing the next challenge.

 

[Freyja]

It seems fair to me that winning one phase excludes you from winning the other.

Trying to be as objective as possible considering I won both phases, I think this is a silly idea. If someone makes a map good enough to be worth winning both phases, better than all the competition because they put the work in, they should win them, and not be automatically ruled out because they did well somewhere else.

Maybe I only feel this way because I don't think contests are about winning (it's nice, of course), and I think anyone entering to win is lost already.

 

[Fruity Snacks]

sure we have individuals who can handle random challenges but mostly tf2m is comprised of people who still need to fully explore the box called "alphas" that contests want them to think outside of. gotta take that into account when designing the next challenge

MvA contest did illustrate this a bit too. One of the next Public Discussions we're doing is relating to contests and how should they be formatted, or even if we should host them. You'll be able to voice your opinions there.

Trying to be as objective as possible considering I won both phases, I think this is a silly idea. If someone makes a map good enough to be worth winning both phases, better than all the competition because they put the work in, they should win them, and not be automatically ruled out because they did well somewhere else.

I agree with this. As I said before, if someone does well, it would be moronic of us to punish them for doing well again. This goes for more than just first place too. We always want to be encouraging everyone to do their best and to keep pushing for their best work, regardless of how other do.

Maybe I only feel this way because I don't think contests are about winning (it's nice, of course), and I think anyone entering to win is lost already.

You're not alone.