Public Discussion: Feedback Quality and Quality of tests

[Turbo Lover]

Do you feel that people are using our 'tests' more as a casual play session, and not treating it as a test?

The purpose behind testing maps is so we can get an idea of how well they play, whether they're fun and balanced, etc. I think what hinders that is when people forget that purpose. When a player's priorities shift from assessing the map's gameplay to being personally entertained, then that's a player who is probably not giving good feedback - if any. It's not always a bad thing, though. I think it's pretty obvious Sel & Trotim don't join our tests because they want to help improve these maps through observational feedback on the map's play. Most of the time, anyway. But when they do grace us with their presence, they tend to be productive members of the team they're on, which facilitates a better testing environment. On the other hand you've got Layl who plays Sticky Jumper/Caber Demo, goes Engineer and tries to set up level threes in front of our spawn door, goes Scout and kills our teleporter entrances and then hides before killing our teleporter entrances again. At best these things don't have any effect on the outcome of a test - positive or negative - and at worst they directly impede testing. Sorry to be dropping names, but I can only draw from experience.

How can we as a community or as individual members of the community promote giving higher quality feedback?

I asked my buddy Noizeblaze - who regularly attends tests but rarely gives feedback - and if his opinion is one that is shared by others, then the problem is that they don't know what to feedback. For both mappers and players, knowing what makes a good TF2 map is important - a lack of this knowledge would make giving feedback on an in-development map difficult. I also asked what could be done to help this issue, and in essence, what would be helpful to these people already exists, and has existed for 5 years. What was suggested were in-depth tutorials explaining fundamental mapping concepts. Kind of like those tutorials grazr wrote. Maybe this only reflects a minority of the people who join our tests but don't leave feedback, but having more tutorials like the ones grazr wrote would be pretty bitchin', either way.

 

[fauks]

A test operator should have specific aspects of the map to test. After 5 minutes, ask how spawn times are working out. After 10 minutes, ask how forward spawns are working out. If that's what the map author needs to know. Yeah there's that generic line in the forum every gameday thread about seeking feedback on specific things, but do authors really request it?

 

[Crash]

I think a thing people should keep in mind is what version the map is on. NOT in the sense that we need to debate if that beta map should have dev textures anywhere or if that alpha should have any detailing (this is a dumb argument, please stop having it), but what you are leaving feedback on.

A map that is just getting it's gameplay finalized doesn't need detailing feedback. A map that is early alpha doesn't need endless !fb clip's.

I think a lot of people want to be helpful without knowing how to help so they fall back on the basics.

The best thing is to just ask the mapper what they want feedback on. There is usually a certain aspect(s) they are trying to test, so find out from them what that is!

 

[Egan]

• How can we as a community or as individual members of the community promote giving higher quality feedback?

Exactly what people were saying in the Showcase Public Discussion, is that people use the maps-that-have-been-showcased threads as inspiration for feedback and information. Not necessarily just because of the gameplay or detail, but because the threads generally had more in-depth feedback in them, in those times. I think observing why this feedback happened naturally and why it's not happening as often now, would be worthwhile. I've analysed it a bit, but I wasn't here during those times, so I'm not really sure if there's anything specific missing. I doubt it's just specific people, it's probably the environment like 'I don't want to give my opinion cause I might be wrong or people will publicly shame my opinion' or 'I'm just too lazy to do it'. Also like Crash says I think some people don't really know how to give good feedback or how to anaylse maps for more indepth kind-of-shit. We mentioned this in the last meeting that I would work on updating Idolon's 'how to give better feedback' thread; I've talked with Idolon, he says that's good, and I'll work on that when I get the chance to do so.

Supporting quality feedback: maybe showing feedback statistics - top 10 in the last month / year / all time or whatever, gamebanana has a system where admins/moderators can give a spotlight recognition on a post for being an 'exemplary post' -example - maybe implementing a similar feature here could be cool.

• How can we as a community or as individuals help other people analyze and understand feedback that we give?

As mentioned, Idolon's feedback tutorial thread, making it more in-depth, it's on my to-do list.

Maybe like 3kilksphilip does for CSGO theory crap, we can do some for TF2 mapping theory crap, like 'here's what important, what you should be looking out for, what you should be asking' - kinda similar to the more in-depth feedback tutorial thread but maybe easier to digest or watch when you're bored or whatever. TF2 theory videos on youtube could be fun to make / watch in general. The tf2maps.net youtube channel can just 'Like' those videos and add them to a playlist or something.

• Do our gamedays really help with getting/giving feedback? Do we need to do more in the gamedays?

I think they give good feedback, and if people wanted more feedback on their map they should either answer my question of what specific feedback they want me to ask about the map when it's being played during the gameday, or to review the STV demo some amount of times until they see what the players do to create such feedback that was posted - understanding context is important in any medium: movies, stories, conversations, feedback, etc.

I've heard we're doing an alright job giving feedback to people in most cases. There are sometimes maps in impromptus that go in and out without a lick of feedback apart from the little things, but in most cases the mapper should be observing those playtests and should be able to tell if its playing like what they thought it was going to play like. And I'm getting into what I was going to write down for the in-depth shit, but I think some people don't understand that feedback is 'feedback to your actions' and not 'what to do next'. Feedback is where players give their observations of how your map plays, you are supposed to know how its supposed to play before the test, and then if it doesn't play how you think it should play, ask yourself why, and then take in all observations when you go to change things again for the next map iteration.

 

[xzzy]

I said it somewhere else around here, but the first thing people leaving feedback should do is shut their mouth until they've played the map 2 or 3 times. The first time anyone plays a map is almost completely useless because everyone is trying to learn the layout and the teams might be comically imbalanced due to a bunch of people stuck in the download.

In-game map theory discussion is pretty useless too, you can't cram much context into a short chat message. Play the map for a while, find the map's thread, and write a couple paragraphs talking about impressions. Annotate with screenshots if needed.

I understand that's a lot of work, but it's also infinitely more valuable to the mapper. It also gives other people from the play test an opportunity to comment on the feedback. This has the added benefit of improving the overall quality of the forum as a mapping resource.


As for the execution of the actual map tests, they're good and bad. Good in the sense that it's the only way for most mappers to get a bunch of human players running around generating data. That's super valuable, but that's as far as it goes.

The bad part of it is the attitude of the people that do show up. I don't consider myself a regular on the tf2maps servers but when I do drop in, the overall attitude is shit. There's always a couple people trolling and being a general pain in the ass, some other people whining about the tf2maps community, and at least once per session some pointless philosophical debate fires up, such as "how can this be a beta map, it has dev textures."

End result is I only show up to play when there's something unique happening, like a judging period of a contest. Because in general it's just not a fun place to hang out.

 

[Fruity Snacks]

As YM mentioned before, the Heatmaps will be a fantastic way to get data and feedback from tests, even with little kills.

I'll be writing up a guide on that soon, so everyone knows how to interpret the data. Should help a toooon

 

[RubbishyUser]

The bad part of it is the attitude of the people that do show up. I don't consider myself a regular on the tf2maps servers but when I do drop in, the overall attitude is shit. There's always a couple people trolling and being a general pain in the ass, some other people whining about the tf2maps community, and at least once per session some pointless philosophical debate fires up, such as "how can this be a beta map, it has dev textures."

End result is I only show up to play when there's something unique happening, like a judging period of a contest. Because in general it's just not a fun place to hang out.

I don't think that's a fair analysis of the testing I see. I'd say that there are only ever 1 or 2 bad eggs who can really make a test poor, "trolling" or even effecting the atmosphere by complaining about the map constantly. If anything, I'd say that when your map gets a weird test where everyone decides to go caber demo, for example, it actually speaks volumes about the map. If you don't think it tells you anything useful, then you could always just do another test the next day!

And personally, I enjoy seeing Layl turn up to tests and trying whatever weird stuff he does to break the map. If his caber demo is seriously a problem, then it's your map has a problem - not Layl. Are you going to say that caber demo's are not part of real TF2?

I said it somewhere else around here, but the first thing people leaving feedback should do is shut their mouth until they've played the map 2 or 3 times.

Not sure I agree with that, particularly because that just allows opinions to calcify before they leave feedback. Way I look at it is like this. Say you identify a problem round 1 - scouts are too powerful. Now normally, there'd be discussion in the chat either confirming your opinion, in which case it probably is a problem, or people disagree with you, forcing you to reconsider round 2. If you all keep radio silence, then even in round 2 you're going to go in thinking that scouts are too powerful, blaming every death to a scout on the map instead of the player. Then by the time discussion starts, everyone has strong opinions that aren't going to be changed by the consensus of their peers. No conclusion is reached, and the feedback is worse for it.



I 100% agree with everything Egan said, and nominate Crash, Grizzly Berry and our other resident mapper/youtubers to start making voice commentaries about map design and feedback. It'd be good for your channel and good for TF2maps. Those of us who want to see more in depth post-mortems of game days for their maps, I suggest making some yourself. I know I'll always reciprocate if someone picks apart mine.

 

[Sergis]

I said it somewhere else around here, but the first thing people leaving feedback should do is shut their mouth until they've played the map 2 or 3 times. The first time anyone plays a map is almost completely useless because everyone is trying to learn the layout and the teams might be comically imbalanced due to a bunch of people stuck in the download.

i disagree

the first time of playing the map is how the server owners will see the map

if a map sucks on the first play they will simply never play it again

 

[Fruity Snacks]

Something I guess I should have mentioned too was that you want to consider your expectations for a test, compared to others.

Some people want to make this a career, so they might have a different expectation than someone who wants to do this type of thing a hobby, or because they appreciate custom maps.


Food for thought.

 

[henke37]

One thing that would help is automated checks. Things like missing cubemaps and such. It's annoying to test something with obvious flaws. And the resulting feedback isn't going to be exciting. "!fb ERROR" and "!fb I don't see any missing texture" is only useful the first time, not the tenth. These errors can even skew the test results in extreme cases. A missing prop can mean a missing route.

But as for actual testing? I don't think there is anything possible to gain from policy changes, we have reached the current limit. The only possible thing would be to run tests on all the servers at the same time to get more players.

 

[takabuschik]

The main problem I have with giving feedback via !fb is that you know most chances are if you give feedback while you are alive you'll dye. If you dye you will have to wait through respawn times, you won't contribute to the team effort to win and worst of all, your coordinates will not be in the feedback.

What we really need is some sort of server plug-in that makes it so when you start typing in chat !fb or !gf you will recieve a special icon over your head that says you're writing feedback, and then everyone will know not to kill you. The icon will NOT make you invincible, so you can't abuse it, but it will make the fact you're providing feedback more noticable. I have no idea whether it is even possible to do something like that, but after seeing all the magic plugins geit made to the server (feedback, heatmaps), I guess it can happen.
I accidentaly kill feedback writers and get killed myself many times while writing, even when I stand at the corner facing the wall, being completely harmless. It's really annoying.

 

[henke37]

The thing is, the server must be told that you are typing. The client needs to do that. All the custom stuff we have on the server is exactly that, on the server.

What you would need is a change to the client. It would be a trivial addition, if you had access to the source code. Which we don't.

 

[Geit]

The thing is, the server must be told that you are typing. The client needs to do that. All the custom stuff we have on the server is exactly that, on the server.

What you would need is a change to the client. It would be a trivial addition, if you had access to the source code. Which we don't.

I once asked Robin if he could add a server event for typing start/stop events, like GMod has. He refused. :c

 

[Fruity Snacks]

I once asked Robin if he could add a server event for typing start/stop events, like GMod has. He refused. :c

Ask again, since they expressed interest in helping mappers, maybe they would.

 

[henke37]

While at it, add an event for when people open the menu.

 

[xzzy]

And personally, I enjoy seeing Layl turn up to tests and trying whatever weird stuff he does to break the map. If his caber demo is seriously a problem, then it's your map has a problem - not Layl. Are you going to say that caber demo's are not part of real TF2?

No, I'm saying the tf2maps servers just aren't fun to hang out and play on. I didn't intend to say anything about whether good information is gathered. It's more about keeping the server populated than anything, you can't really call it a proper playtest if there's less than 10 people playing.

Not sure I agree with that, particularly because that just allows opinions to calcify before they leave feedback. Way I look at it is like this. Say you identify a problem round 1 - scouts are too powerful. Now normally, there'd be discussion in the chat either confirming your opinion, in which case it probably is a problem, or people disagree with you, forcing you to reconsider round 2. If you all keep radio silence, then even in round 2 you're going to go in thinking that scouts are too powerful, blaming every death to a scout on the map instead of the player. Then by the time discussion starts, everyone has strong opinions that aren't going to be changed by the consensus of their peers. No conclusion is reached, and the feedback is worse for it.

I'm just suggesting people learn to separate a knee-jerk reaction from useful criticism, and if maintaining silence for a couple plays is what it takes, then so be it.

 

[Fruity Snacks]

And personally, I enjoy seeing Layl turn up to tests and trying whatever weird stuff he does to break the map. If his caber demo is seriously a problem, then it's your map has a problem - not Layl. Are you going to say that caber demo's are not part of real TF2?

They're a problem on a lot of official maps and custom maps. It's hard to not have them be a problem. It's all down to the skill of the player.

 

[SSX]

Personally to me...I'd like to see some sort of compliments made on the map. Because if it's all feedback on the map telling how to fix stuff, where's the motivation for some of us to continue mappin? I don't know if this is something only I get discouraged with, but I'd like to be able to see some form of a figurative thumbs up, or 'Good Job' somewhere in there.

We're not all Aly, Crash, Frozen, Berry, or other great mappers. Even if they do go through a lot of feedback just to make their maps look, and play good. Not all of us are capable of doing it like that, and some of us just want the satisfaction of "I made my own map." but others beat on us for it because it doesn't follow a formula. I don't ask for perfect feedback, I want feedback, but I don't want to feel like nothing works. I would just like to see at least a comment in there like; "I really like what you've done with this area." or "I like the route you made here." This is why I've been dragging my feet lately because I don't feel motivated to actually keep mapping for TF2 if this keeps happening...

 

[henke37]

That's a problem that all feedback systems have. People tend to comment on things they hate more than things that are satisfactory. You need something truly outstanding to get positive remarks.

 

[xzzy]

Consider it artist training, anyone who does anything creative needs to find a way to persevere and push for their goals even if all they receive is criticism.

No one will ever give it to you, you gotta take it.