Mappers Vs Machines: The Results

[pennsyrail]

Y' know, I think this contest was responsible for exposing and encouraging several things. First off, there are REALLY good mappers, and it's because of them that this site was founded and because of them that we mappers have a place to turn to for inspiration and ideas. We've got so much here that I don't think we should complain too much about using a site that's not ours and not getting things our way. That being said, maybe next time a contest that doesn't prevent professional mappers from joining but doesn't allow their entries to be judged alongside the others? Maybe you guys are fine with the way it works. Either way, it is great to see how much comes out of these contests. Tf2 may be dying, but mappers certainly aren't

 

[FishyUberMuffin]

The rules were that no one map (not mapper) could win more than one of the four basic prizes, as well as the best overall prize. I definitely see your point: in future I think i'm going to tweak the rules so this won't happen again, but it would be pretty shitty to do that mid-contest so that one person didn't win Too Much.

I knew it. I was not the only person who thought that a person winning 3 out of 5 category was really unfair. Good job on point that out Izotope.

 

[Crash]

That being said, maybe next time a contest that doesn't prevent professional mappers from joining but doesn't allow their entries to be judged alongside the others? Maybe you guys are fine with the way it works. Either way, it is great to see how much comes out of these contests. Tf2 may be dying, but mappers certainly aren't

The problem with this is how do we draw the line and define it? Maybe you get bumped up if you've ever gotten paid for mapping? Well that's easy to lie about, and even if you don't should we count the smaller trade map jobs in along with getting paid by Valve on a map? Or say just if you're a published mapper in TF2? Well what about all the other extremely talented mappers who haven't been published but are on that same level? Do we let them still compete with everyone, despite being just as skillful? Or if we just tried to base it on talent/ skill in work, who's do decide that arbitrary concept? Will it be fair to those we DO decide are up there, who didn't want to be judge along side the top-tier work? Or will it be insulting to those we DON'T put in that category?

Then there's the issue of how we'd divide the prizes up. Does the "top-tier mapper" category get a better prize, or the same? Do we have to now double all of our prizes to cover it?

What it boils down to is, should we punish and segregate skilled mappers arbitrarily because they have more talent... or should we keep it open to everyone and let the best end result stand out on it's own?

I understand where you guys are coming from, when I started out it was just as frustrating seeing veterans doing this great work, and I felt like I'd never reach that point. You just have to keep at it and learn everything that you can from each project that you make.

 

[Fruity Snacks]

The problem with this is how do we draw the line and define it? Maybe you get bumped up if you've ever gotten paid for mapping? Well that's easy to lie about, and even if you don't should we count the smaller trade map jobs in along with getting paid by Valve on a map? Or say just if you're a published mapper in TF2? Well what about all the other extremely talented mappers who haven't been published but are on that same level? Do we let them still compete with everyone, despite being just as skillful? Or if we just tried to base it on talent/ skill in work, who's do decide that arbitrary concept? Will it be fair to those we DO decide are up there, who didn't want to be judge along side the top-tier work? Or will it be insulting to those we DON'T put in that category?

Then there's the issue of how we'd divide the prizes up. Does the "top-tier mapper" category get a better prize, or the same? Do we have to now double all of our prizes to cover it?

What it boils down to is, should we punish and segregate skilled mappers arbitrarily because they have more talent... or should we keep it open to everyone and let the best end result stand out on it's own?

I understand where you guys are coming from, when I started out it was just as frustrating seeing veterans doing this great work, and I felt like I'd never reach that point. You just have to keep at it and learn everything that you can from each project that you make.

To piggy back, you really should not be looking at it as "these people who are better than me are preventing me from winning" - thats wrong they are doing nothing to prevent you from winning.

Really (again) should be looking at these people and going "what can I learn from them" or "How can I improve to do as well as them." You walk around life thinking that people who are better than you are preventing you from doing things, you're never going to improve. It's not like they're taking up spaces and when they leave, you can fill their space. You can move forward, move up, become one of those skilled mappers.

Also, for what it's worth - I do professional level design now and the last contest I entered (dyn PL), I came in 9th.

 

[BigfootBeto]

Best Overall was the best on average across all voters and categories. Another map might do better in one thing (and so win a prize in it) but worse in everything else, and so have a lower average score.

I did the averages of each category like you said, but even the "Overall" numbers for each category don't match. I'm guessing each category was weighted, but I don't know by how much. What was the weighting for each category?

 

[LeSwordfish]

I'm really not sure why people are going through and checking everything - I did, like, do maths for this? It took quite a while? It's not just scribbled on the back of an envelope? It seems a bit like you (general you, another person is doing this via PM with me) want to take me or Freyja down a peg or two. Thanks for all your help and support, I guess. When the end result to a contest is overwhelmingly nitpicking and complaints, why should we bother running them?

That said, the weightings are below:

• Everything in the Judges scores is weighted at 10%, except Aesthetics and Layout which are 15%
• Everything in the public scores is weighted at 20%, except Aesthetics and Layout which are 30%
• Best Overall is the best average of the overall public score for a map, and the overall judges score (weighed 50/50 public/judges).
• Best Aesthetics was the same - 50% Judges Aesthetics, 50% Public Aesthetics
• Most Creative is the average of the creativity scores for Judges and Public, again weighed 50/50, (with Judges score split 50/50 popfile/layout).
• Public Favourite is the highest overall public score.
• Most Fun is Layout and Popfile combined for public and Layout and Popfile Fun combined for Judges. The weightings from before don't continue here - it's 25/25/25/25.
• These weightings were decided by myself and hydrogen midway through the contest.

That gets us numbers that look like this:

 

[Another Bad Pun]

I'm disappointed that so many people are blaming others for their losses instead of taking accountability for what they made.

This is a contest, not everyone is going to win, because not everyone can make Teien, or Steep, or Airdrop. The negative reaction these results are still receiving shows a real lack of sportsmanship, and I honestly think it's pretty shameful.

 

[Viemärirotta]

I'm disappointed that so many people are blaming others for their losses instead of taking accountability for what they made.

This is a contest, not everyone is going to win, because not everyone can make Teien, or Steep, or Airdrop. The negative reaction these results are still receiving shows a real lack of sportsmanship, and I honestly think it's pretty shameful.

Exactly. Contests are a great way to display your skill and creativity, don't go all out on others for not winning, just because you didn't do as good as others.

 

[Hyperion]

I recalculated all the results. The first score number is the real and the second number used for judges overall scores. The first result number is the announced result and the second the real result. Following errors were found:

• Egan's overall score for Barren got changed from 8,8 to 8,9, resulting judges overall score for Barren to drop from 8,117 to 8,083
• Egan's overall score for Calico got changed from 7,9 to 8,0, resulting judges overall score for Calico to drop from 7,650 to 7,617
• Egan's overall score for Factory got changed from 5,05 to 4,65, resulting judges overall score for Factory to raise from 5,867 to 6,000
  
• Egan's overall score for Havana got changed from 7,95 to 8,15, resulting judges overall score for Havana to drop from 8,117 to 8,050
  
• Egan's overall score for Relic got changed from 7,35 to 7,45, resulting judges overall score for Relic to drop from 6,833 to 6,800
  
• Egan's overall score for Seabed got changed from 9 to 9,1, resulting judges overall score for Seabed to drop from 7,500 to 7,467
  
• Egan's overall score for Sequoia got changed from 9,45 to 9,65, resulting judges overall score for Sequoia to drop from 8,917 to 8,850
  
• Egan's overall score for Steep got changed from 9,45 to 9,65, resulting judges overall score for Steep to drop from 8,600 to 8,533
  
• Egan's overall score for Teien got changed from 8,35 to 8,65, resulting judges overall score for Teien to drop from 8,583 to 8,483
  
• Egan's overall score for Waterfront got changed from 8,25 to 8,35, resulting judges overall score for Waterfront to drop from 7,658 to 7,625

Due to the nature of found errors, I hardly believe it was copy-pasting error and I have to think that the published scores of judges changed after publishing.

• Overall Public Fun results are calculated from Layout and Creativity, not from Layout and Popfile as the weighting post suggests:

That said, the weightings are below:

• Most Fun is Layout and Popfile combined for public and Layout and Popfile Fun combined for Judges. The weightings from before don't continue here - it's 25/25/25/25.

• Because of that, all Overall Public Fun scores were changed a lot, for example Airdrop raised from 5,227 to 5,909 while Dockyard dropped from 7,611 to 6,834.
• Best Overall results did change because of previously mentioned disparity in Egan's results. Factory was the only one that lost points in published results and 9 other maps gained around 0.01-0.05 points. However, there was no place changes.
• Because of the ambiguity in Overall Public Fun score, the Most Fun scores changed too but there was no change in winner, Sequoia still won with the score of 7,93 and Steep was 2nd with the score of 7,88

The spreadsheet I used and where I copied and checked all values can be found here.

 

[Hyperion]

I'll make a separate second post because this will be just observations based on my nitpicking mind.

• The Most Fun winner Sequoia had big difference in public and judge scores, 6,86 vs 9,00. The highest public fun score Dockyard got 7,61 from public but only 6,83 from judges
• The Most Creative Airdrop tied Sequoia in both public and judges score but because of the order of awards given, Airdrop got it. If Sequoia would have won Most Creative, Most Fun would have been Steep. If Public Choice was done before that, Most Fun would have been Barren.

So, actually the list of winners with different order would have been

• Most Fun: Barren
• Best Looking: Teien
• Most Creative: Sequoia
• Public Choice: Steep
• Best Overall: Sequoia

Or if Most Fun was before Public Choice

• Most Fun: Steep
• Best Looking: Teien
• Most Creative: Sequoia
• Public Choice: Dockyard
• Best Overall: Sequoia

Basically Barren, Dockyard and Steep fought for an award. This raises the question if one map actually should have won multiple categories to eliminate such situation that is shown above. The results would be following

• Most Fun: Sequoia
• Best Looking: Sequoia
• Most Creative: Sequoia
• Public Choice: Sequoia
• Best Overall: Sequoia

Of course no one would want that result from the contest and therefor I think the Best Overall should have been done first by taking average of each award categories, Public Overall and Judges Overall. Then exclude the Best Overall from other categories. Then the list would be following

• Most Fun: Steep
• Best Looking: Teien
• Most Creative: Airdrop
• Public Choice: Dockyard
• Best Overall: Sequoia

Or if Public Choice was first to count

• Most Fun: Barren
• Best Looking: Teien
• Most Creative: Airdrop
• Public Choice: Steep
• Best Overall: Sequoia

These results might be different without corrected points but because it wouldn't matter, I don't want to use my time to check that.

 

[LeSwordfish]

se results might be different without corrected points but because it wouldn't matter, I don't want to use my time to check that.

yeah because you didn't want to waste your fucking time now did you

• Egan changed his scores fairly late in the day. The ones the judging used were correct.
• The Most Fun didn't change, though yeah there was an error there.
• I chose to prioritise Public Choice as different because it seemed to me that of all the prizes, it was the one for which it was important to be different to Best Overall.
• So nothing changed.
• Congratulations to everyone for putting so much effort into spoiling this!
• I'm locking this fucking thread.

 

[Crash]

Oh boy let's just get right on top of organizing the next contest. That sounds like a great time!