Featured Maps - I think the selection is biased... let's discuss about it

[Midlou]

Very important disclaimer before continuing reading what I have to say:​

I don't want to cause intrigue; I just want to start an important discussion that, in my opinion, is in the best interest of this community.

Let's talk about the elephant in the room. How did the community come to accept the staff defining 'featured maps' in a black box, without a well-defined criteria of what gets selected? As clearly explained in this thread here: '[...] maps will be nominated anonymously by the public, and then reviewed internally by the staff team.' So, 'reviewed internally' means some undisclosed criteria are used for map selection. I don't know about you, but when a staff member gives 'featured map' status to their own projects, it's AT LEAST weird.

Getting your map accepted on the 'featured maps' list is no small feat. This list gives visibility to your project, serves as a respectable portfolio. Unfortunately, it seems like some members of the community are more equal than others and are getting way, WAY more attention than they deserve.

In summary, I want to propose a discussion to figure out if this whole 'reviewed internally' thing still makes sense.

Maybe it's time to establish a clear, well defined criteria for which maps get selected.​

Okay, you disagree with everything I said.

At the very least, the staff could explain the current method of selecting featured maps.​

Honestly, right now, it looks pretty shady.

Edit1:
Here are the links to the discussions that took place on Discord:

TF2Maps Discord: https://discord.com/channels/217585440457228290/217586062736883722/1191716124761542657
UEAKCrash Discord: https://discord.com/channels/281772513384726528/281772513384726528/1191718466462425118

I think it's important to keep these discussions accessible; Discord isn't very good for historical purposes.

Edit2:
After discussing with community members, I believe the thing about bias is not relevant and should not be the focus.

Spoiler: Discussion about bias that is probably not relevant

So, I know what I'm about to question might sound like heresy to some members of the community, especially since a few of them benefit from this trend. Please approach this discussion with an open mind; it's not a personal attack on anyone, even though the motivations are personal.

After checking out the 'featured maps' list for the year 2023, I decided to do a quick dive into the connections of community members whose maps got selected. What I found is at least intriguing:

Spoiler: Data

Staff: 2
Ex-staff: 10
Moderator: 1
Has staff on team: 2
Friend of staff: 17
Inactive: 2
Normal: 11

Empire - youporkchop5 (moderator)
Logwash - Rhamkin
Snowbank - iiboharz (ex-staff, friend of staff member)
Undergrove - Sweepertank (friend of staff member)
Fjord - Fiddleford
Megasnow - Berry (ex-staff, has staff member on the map team)
Camp Saxton - Void (ex-staff, friend of staff member, has staff member on the map team )

Control Points:
France & Dark Marsh - Panckakebro
Holyhell - Panckakebro
Nixie - Idolon (ex-staff, friend of staff member)
Villa - Kevin Gator
Mainline - S.W.A.T.Y (Inactive member since 2010)
Mannbase - skaz_
RocketForge - Nesman (staff member)
Cold Era - Emil_Rusboi (friend of staff member)

Payload:
Vineyard - Panckakebro
Effigy - fubarFX
Saxton Hills - Ravidge (ex-staff)
Extinction - heyyou (friend of staff member)
Fountain - Idolon (ex-staff, friend of staff member)

Arena / Vs. Saxton Hale:
Outlook - LizardOfOz & Phe (friend of staff member)
Hardhat - Penguin (inactive member since 2021, friend of staff member)
Snowcastle - Idolon (ex-staff, friend of staff member)
Facility - Phe (friend of staff member)

Player Destruction:
Tomb - Suna (staff member at the last moment of the selection of featured maps, friend of staff member)
Kodama & Shogatsu - Freyja & Void (ex-staff, friend of staff member)
Circus - Diva Dan (ex-staff, friend of staff member )

Capture the Flag:
Sidewinder - Stickface ( friend of staff member, promoted by UEAKCrash )
Toucan - Suna (staff member at the last moment of the selection of featured maps, friend of staff member)
Terminus - Stack Man (friend of ex-staff member)

Mann Vs. Machine:
Hideout - Mikroscopic ( friend of staff member )
Humbridge - Elizabeth
Whitecliff - Elizabeth

Bonus:
TF2 Party - BigfootBet
Staff: 4.65%
Ex-staff: 23.26%
Moderator: 2.33%
Has staff on team: 4.65%
Friend of staff: 39.53%
Inactive: 4.65%

Normal: 25.58%

79.07% somehow related to staff of TF2maps community.

With these values in mind, I believe the trend in the 'featured maps' list of 2023 is this: It pays off to be friends with the king. Seriously, two selected maps are from the staff member who led the nomination threads on the forum.

The meme of Obama awarding himself has never been more real.​

In this thread here, it was mentioned that the maps would be tested on a gameday. I don't know if the demos were recorded on that day, but I couldn't find my map koth_gamma on the list. I believe the same might have happened to most maps, considering there was very little time to test all the nominations that the community suggested.

Basically, it seems like if a map nomination wasn't from someone with a connection to the staff, the map got quickly disqualified.

Now, I think it's fair to disagree with the data I've put together. Some connections are kind of ambiguous, and my selection criterion was based on interactions with staff members. I considered whether they have any staff member on their Steam friends list or on the forum. I also took into account the number of interactions with any staff member. If a member got a happy birthday from a staff member, I counted it as a connection. Nobody wished me a happy birthday on the forum, so for sure the staff isn't wishing everyone a happy birthday. One thing is for sure, you can't argue about the quantity of ex-staff members.

Despite so many maps, are former staff members more likely to have their maps selected?​

I suggest that you, the reader, do your own research. if you weren't an active member, a friend of the staff, or somehow connected to the staff, your chance of getting a nomination accepted was low.

 

[Gigachad gaming]

obamna

 

[Lacry]

Like Katsu said in the original thread, I think there is more maps skins than it should, I mean, they look great and all, but these maps are already in game, and featured in this website, do they need to have the skin version featured as well?

Like you said the featured maps are there to get more visibility (and also because their quality of course), and for small mappers it gets harder and harder for people to notice your stuff when the popular ones get all the chances to get more visibility. All things consider I'm a small mapper here, my stuff doesnt even have 200 ratings in the workshop, and this kind of things would help to get more visibilty (Seashore is a featured map, and it's my most popular one), right now I feel like if a more popular mapper drops a map the same week as I do, I'm done for.

This of course is the first attempt of the website doing this, so I hope for next year they can improve it, maybe telling us the factors they choose, and also having the possiblity to feature non-maps stuff, I wanted to nominated my Every Game Mode Prefab, cause I think its very useful and most people is not even aware it exists.

 

[nesman]

I just woke up and haven't read any of this.

 

[nesman]

Please keep in mind I was semi on vacation and not overseeing this entirely.

staff could explain the current method

Quoting directly and it's been publicized by another person this morning as well:

• needs to look good and finished
• needs to play reasonably well
• needs to not be too outrageously non-tf2-themed
• needs to be finished and not still be in active development
• needs to be in some broad way representative of TF2maps and not drag us in the dirt

I want to explain that last one because someone is going to intentionally try and take it the wrong way. Recently (I think 6ish months ago) we found out a mapper turned out to be a pedo/child predator. We ended up removing all their maps from the featured section because it's absolutely not a good look to have that kind of person being featured. So that bullet point was to try and avoid that.

I considered whether they have any staff member on their Steam friends list or on the forum

There are people who are on my "friends list" who I do not interact with anymore. Some staff members might have hundreds of people on their friends list where they really only interact and consider a small group of like, 5, to be their "friend/buddy o pal". I really don't like that my friends list is being reviewed when I don't interact with the most of the 89 that are on there.

the map got quickly disqualified

That is not the case, at least with how I did the few maps I was able to do. I even avoided maps I knew I would have a bias on like my own and said this is someone else's problem. We even shot down some maps by staff/mods.

black box

Some discussions need to happen behind a close door. This was a huge undertaking with the large amount of maps we have on the site, we're bound to miss something. We're more likely to upset more people if we're open about discussion about a map, let's say yours, with many other community members chiming in. What if all those members said the map was bad? Wouldn't make you feel good would it. This is why it needs to happen behind a closed door. Sure the criteria could be more public, I won't deny that.

maps would be tested on a gameday

Please keep in mind I was semi-away. I am quoting the original thread about this here: "If we are uncertain of a map's quality, it will be ran in a gameday prior to the announcement of this year's featured maps."

It's also worth noting we had literal hundreds of maps submitted this year. That's a LOT of maps!

Unfortunately it seems like the discussion turned into more of a "trying to prove bias with a broad brush" and not a "we need more transparency around this". I understand the points about transparency to make it as level of a playing field as possible, but we're bound to piss someone off no matter what we do.

There was also a good discussion in Crash's discord about this where good points were made, I don't feel I need to reiterate them.

I'm sure I'm leaving importing points about this whole discussion out or am bound to piss someone off, sorry in advance, I woke up to all of this controversy and am still functioning on lack of sleep from fighting a structure fire 2 days ago. Join your local department if they're volunteer, we need people in the field

 

[Maid]

Please keep in mind, per 'needs to be finished and not still be in active development', that is probably the explanation for why your map koth_gamma in particular was not chosen to be featured. It's B4, it's not finished, it's not qualified to be featured.

 

[nesman]

Please keep in mind, per 'needs to be finished and not still be in active development', that is probably the explanation for why your map koth_gamma in particular was not chosen to be featured. It's B4, it's not finished, it's not qualified to be featured.

Yes and no. Rocketforge is still technically in "beta" but still got featured, but I haven't touched it in literal years. We also look at last updated dates to see if it's still in active development.

 

[Midlou]

Fine, I'll separate the part of the text that comments on bias because, as I said before, for some, this may sound like heresy. I won't remove this part of the text because I WON'T CENSOR MYSELF, especially when everyone commented on how I shouldn't question if there is bias, even when no one has proven that there is no bias.

But I get it, the data I raised probably doesn't indicate anything concrete, I understand. Anyway, about the issue I want to focus on, transparency in the selection criteria:

• needs to look good and finished
• needs to play reasonably well
• needs to not be too outrageously non-tf2-themed
• needs to be finished and not still be in active development
• needs to be in some broad way representative of TF2maps and not drag us in the dirt

I want to explain that last one because someone is going to intentionally try and take it the wrong way. Recently (I think 6ish months ago) we found out a mapper turned out to be a pedo/child predator. We ended up removing all their maps from the featured section because it's absolutely not a good look to have that kind of person being featured. So that bullet point was to try and avoid that.

That is indeed something, and I would like this to be expanded into a more robust process and preferably one that allows people to see what disqualified their work that took years to complete.

Some discussions need to happen behind a close door. This was a huge undertaking with the large amount of maps we have on the site, we're bound to miss something. We're more likely to upset more people if we're open about discussion about a map, let's say yours, with many other community members chiming in. What if all those members said the map was bad? Wouldn't make you feel good would it. This is why it needs to happen behind a closed door. Sure the criteria could be more public, I won't deny that.

but we're bound to piss someone off no matter what we do

As I mentioned in the Discord discussion, hiding the deliberations/feedback on why a map was disqualified is the wrong reaction. Frustration is part of constructive feedback, hiding it from people builds nothing. Sorry, it does build something; it builds an environment where no one knows why a particular map was selected to receive permanent spotlight on the front page of the forum. So I don't know how everyone finds it unreasonable to question if there's a bias, when no one outside staff knows anything about the selection process.

Please keep in mind, per 'needs to be finished and not still be in active development', that is probably the explanation for why your map koth_gamma in particular was not chosen to be featured. It's B4, it's not finished, it's not qualified to be featured.

It's not about my map; it's about all the maps. Anyway, beta maps have been accepted into the featured list. Hell, beta maps have been accepted for the game itself. If there were a clear definition of the selection criteria, this confusion wouldn't happen (about featured maps, I don't expect a selection criteria from Valve).

 

[Tiftid]

I'm not going to say that the selection this year was biased, because that would make me look like a sore loser (supposing anyone even nominated my maps!). But I do think the process easily could be made more transparent with minimal extra effort from the judges. Here's how:

Some discussions need to happen behind a close door. This was a huge undertaking with the large amount of maps we have on the site, we're bound to miss something. We're more likely to upset more people if we're open about discussion about a map, let's say yours, with many other community members chiming in. What if all those members said the map was bad? Wouldn't make you feel good would it. This is why it needs to happen behind a closed door. Sure the criteria could be more public, I won't deny that.

People have said foul things about me and my maps before, but I've survived. I know that's not true for everyone, but I believe that if I know I can take criticism (constructive and otherwise), I should have the right to submit my map with the stipulation that its judging process will be fully public.

Quoting directly and it's been publicized by another person this morning as well:

• needs to look good and finished
• needs to play reasonably well
• needs to not be too outrageously non-tf2-themed
• needs to be finished and not still be in active development
• needs to be in some broad way representative of TF2maps and not drag us in the dirt

This is, in my opinion, outrageously vague.

• For "looks good and finished", I sort of understand it being subjective more, since I think most people tend to agree on what looks good.
• What does it mean for a map to play "reasonably well"? Are there any objective, statistical definitions you use to decide that?
• "Not being too outrageously non-tf2-themed" is again very subjective. Many maps have been added to the game recently which I would consider pretty outrageous. Those maps then go on to be featured - and conversely, some maps went from being featured to being added.
• I don't really understand why an unfinished map shouldn't be featured if it's already awesome. Sure, it could be changed down the line to not be awesome anymore, but that applies to any map - a map that can be declared "finished" in the short term could still have its mapper eventually decide to revisit and make major changes to it. Furthermore, if this is being used to disqualify alpha maps cause people wanna visit this site for Valve-quality maps or whatever, what differentiates this criterion from the first one?
• The final one is more understandable with your text explanation.

I would also like to point out that contest judging, since 2009, has had a very clear set of rules, and a fully public voting process in addition to private judging. It wouldn't kill you to have a public "yes/no" strawpoll for each of your finalised picks before you actually feature them, so the community has some agency in the maps that are gonna, as you say, represent them.

 

[Yrr]

shockingly the ppl who make good maps tend to end up being notable in the community, either becoming staff or befriending staff

that said, there's some whacky choices here, and its really not a good look to Feature a map by someone whom is banned from the site

 

[Katsu! :3]

and its really not a good look to Feature a map by someone whom is banned from the site

good map is good map, I think saying stuff like this is rather arbitrary considering people only really see the map and not the mapper. Other than that I pretty much agree, I think basing the study on who is friends with whom is kinda stupid when you would effectively get the same data with who is staff, who is VIP, who is neither.

 

[Tumby]

If only finished maps are added, and maps need to be nominated and stuff, then why is longboy featured?

 

[Miki]

I never thought the featured maps were biased, just an honest effort to represent the site with its outstanding and creative maps. They are also good for server owners looking to add top picks for their server. Many entries having some connection to staff I figure comes from being involved in a community, it seems natural many featured maps come from people invested and active in their community.

In addition, the few times I have interacted with staff, they were quite cordial and I didn’t have the impression they harbored bad traits to rig a map list. Also..

If a member got a happy birthday from a staff member, I counted it as a connection. Nobody wished me a happy birthday on the forum, so for sure the staff isn't wishing everyone a happy birthday. One thing is for sure, you can't argue about the quantity of ex-staff members.

Happy late birthday.

Spoiler

 

[Yrr]

good map is good map, I think saying stuff like this is rather arbitrary considering people only really see the map and not the mapper.

I think it matters in the context of being a showcase of work a community has created - to not feature the work of someone whom was Removed from said community as an example of its work.

 

[Maid]

It's also fair to consider that those who become staff tend to be those who are well-known, well-connected and experienced in the community, meaning they most likely get along with the active community mappers here. We're not as large a community as we might seem, and the Staff are all pretty friendly - everyone knows everyone here, once you've been around long enough.

Staff interacting with and knowing the mappers who have their work featured makes sense, because those mappers are likely also community veterans who've spent a long time honing their craft, getting help from and giving help to others in the mapping scene. For this reason it doesn't make much sense to consider which mappers who've had their work featured are friends with staff, since there's plenty of mappers who are friends with staff who haven't.