Public Discussion: Feedback Quality and Quality of tests

Discussion in 'Site Discussion' started by Fr0Z3nR, Jul 30, 2014.

  1. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,902
    Hey guys!

    So the previous Public Discussion is still going and you're welcome to forever discuss it over in that thread. However, I'm going to start up the next discussion which is on "Feedback" and how the community can give better feedback overall.

    I will be blunt, but a lot of people have complained about TF2Maps.net and our feedback quality. It's either good, or bad and unhelpful... with the "bad and unhelpful" popping up far more often than the good. This is a natural issue with giving feedback and analyzing it, but finding ways to promote giving more good feedback, or even a discussion on how to teach people how to analyze feedback could go a long way.

    So, some things to consider when you think about this (and, these aren't the only things you can touch on, if you feel there is other stuff you want to talk about).
    • How can we as a community or as individual members of the community promote giving higher quality feedback?
    • How can we as a community or as individuals help other people analyze and understand feedback that we give?
    • Do our gamedays really help with getting/giving feedback? Do we need to do more in the gamedays?
    • Do you feel that people are using our 'tests' more as a casual play session, and not treating it as a test?
    • Is there other tools besides the Feedback system and Geit's heatmap plugin that we could use to help raise the quality of our feedback?

    There's just some ideas on things to chat about, but feel free to elaborate on whatever you think is relevant.
     
  2. RubbishyUser

    RubbishyUser L7: Fancy Member

    Messages:
    414
    Positive Ratings:
    242
    From the feedback I'm getting on Octagon, I'm realizing that unhelpful feedback is often no fault of the player themselves - by being inside the fight, they are quite literally unable to see the bigger picture.

    For example, in a1, one of the doors is rendered inaccessible by an arrow that appears that directs players towards a shortcut that opens up when the point is capped. During the imp a couple players complained that I'd cut off their options and that the other arrows present in the area were enough to redirect players regardless. By already knowing what to do, they were unfairly biased after only 10 minutes of play.

    In a2, I removed the arrow and players determinedly took that door, ignoring all other arrows they passed on the way there - before complaining about walk times!

    As a result I don't think that the feedback itself could be much improved without educating testers themselves on exactly how their vision is clouded when playing. In other words, you can learn how to give better feedback, but tf2maps.net can't bestow it upon us.

    On a different note though, would it be feasible for the !fb feature to record your location and direction of facing when you start typing over where you hit enter? This'd solve problems with being killed mid feedback if possible.
     
    • Thanks Thanks x 2
    Last edited: Jul 30, 2014
  3. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,739
    What? There is -nothing- I hate more than seeing 3 people on one team walking around a map looking at things to leave !fb messages about while the other team is all active. Completely throws team balance and completely throws the percieved map balance.
    We should be casually playing maps at tests. If we don't play them like 'normal' players in tests, that is testing.

    Once we start really using Geit's heatmaps as a community, there will probably be an increase in feedback quality, even if 'feedback' is simply heatmap data. We can use the heatmaps to point out areas to the developer and developers can see their own data and draw their own conclusions.
     
    • Thanks Thanks x 5
  4. DonutVikingChap

    DonutVikingChap L5: Dapper Member

    Messages:
    226
    Positive Ratings:
    142
    As for point #3 and #4, I think gamedays are great and provide extremely helpful playtesting & feedback, and actually find it positive that some people use it more like a "casual" play session, since it means people play like they normally would so you get more genuine playtesting.

    I obviously see the problem in that they don't give a lot of valuable feedback then, but I personally think the pros of "casual" testing, as compared to a very strict testing environment, outweigh the cons.

    This doesn't mean there could be good, non-intrusive-on-the-current-gameday-system ways to encourage good feedback, though. I don't really have any great ideas for it, but perhaps you could let map makers "thank" people's feedback on the feedback page or something like that.

    Also, some kind of official "feedback tutorial/guidelines" on the forum could maybe be helpful.
     
    Last edited: Jul 30, 2014
  5. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,739
    Sadly Impossible.
     
  6. DonutVikingChap

    DonutVikingChap L5: Dapper Member

    Messages:
    226
    Positive Ratings:
    142
    How about typing only "!fb" in the chat first and hitting enter, and then typing your message afterwards?

    Like the chat would say in green text or something "Please enter your feedback" after you type "!fb".
     
  7. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,130
    1 - idk if we can apart from outright paying for feedback and thats a horrible idea. if people cbf typing the feedback up there isnt much to make them do that. besides, low quality feedback is good too as it is closer to what random pubbies would think of the map should it make its way onto servers

    2 - hammer into their heads that stv is the best feedback there is. watching what players do and trying to understand why they do what they do and not what the map creator wants them to do is quite important for becoming a better mapper. i think i read somewhere valve tested their stuff that way too - by having the playtester just play the game and the devs sitting quietly behind him facepalming. some guide "The Dude That Only Said Your Map Sucks And You" about understanding people might help

    3 - yes, we get the stvs. doing something special in the gamedays would probably detract from actually playing the map

    4 - yes. i have noticed conga lines forming during nonsetup time. it pretty much is "play some tf2 on a fresh layout" while it should be between that and a srstest. ideally people should play casually, but without outright fucking around. specific tests would be interesting to see maybe such as how the map handles 7 heavy medic stacks or five wrangleneers or a scout rush

    5 - idk it seems enough for me to get the info i want. feedback system could use annotation stacking and feedback message identifier that doesnt require me to press shift like ! and some way around the getting killed thing. heatmaps now that ive tried them out while nice and cool seem rather useless tbh. our alphas generally get updated after a single test and that doesnt give a useful sample size. by the time map is cool and familiar and a rc and people are willing to sit 3 hours on it its already been played and tested and turned out okay even without heatmaps.
     
    • Thanks Thanks x 2
    Last edited: Jul 30, 2014
  8. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,130
    could it instead on hitting enter grab the place and direction where you were killed if you are dead

    then again it wouldnt work right if you manage to respawn while typing
     
    • Thanks Thanks x 1
    Last edited: Jul 30, 2014
  9. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,739
    Use:
    Then do a second !fb or !gf with your feedback.
     
  10. DonutVikingChap

    DonutVikingChap L5: Dapper Member

    Messages:
    226
    Positive Ratings:
    142
    Yeah I mean actually integrating that mechanic to the system to make it show up as a single feedback so it's less confusing.
     
    • Thanks Thanks x 1
  11. Idolon

    aa Idolon the worst admin

    Messages:
    1,421
    Positive Ratings:
    4,154
    Totally agree. Geit, do you think it would be possible to have a download link to the demo from the feedback page? I don't even know how to access the demos as it is right now.
     
    • Thanks Thanks x 7
  12. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,739
    Oh hey how long as the list of people thanking a post been colourized?
     
  13. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,902
    Since the site got moved to a new server a week and a half ago.
     
    • Thanks Thanks x 1
  14. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,130
    http://demos.geit.co.uk/
     
    • Thanks Thanks x 1
  15. RubbishyUser

    RubbishyUser L7: Fancy Member

    Messages:
    414
    Positive Ratings:
    242
    On that note, could that colouring be a little more washed out and whiter across the board? It's a nice idea, but I'm finding it's a little annoying to scroll past a couple of posts and see a rainbow of colours as people thank other people. It's slightly distracting.
     
    • Thanks Thanks x 1
  16. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,902
    Maybe.

    Lets move this back to map feedback, not site feedback :p
     
  17. RubbishyUser

    RubbishyUser L7: Fancy Member

    Messages:
    414
    Positive Ratings:
    242
    OK Frozen.

    I for one, would really enjoy it if players gave more in depth feedback inside the threads themselves. It takes more time, but it's far more useful than a single !gf - especially if you're complaining about the map. Telling someone you aren't having fun and nothing else, is not only poor feedback but it's poor courtesy if you don't at least try to explain why.

    Actually, it just occurred to me that the TF2maps servers should have a little thing appearing in chat when you join that gives you the instructions on using !fb and proper feedback behavior. If it is there, then clearly it isn't doing it's job and maybe we should step up to those pop-up box things like what is used to votemap.
     
  18. Geit

    aa Geit 💜 I probably broke it 💜

    Messages:
    597
    Positive Ratings:
    1,110
    Done.

    About 2 days

    It's there. If you've claimed your map on the feedback panel then it'll spam the chat 120 seconds after a player joins with a brief overview of the system and a prompt to type !fh (feedback help)
     
    • Thanks Thanks x 2
  19. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,130
    can you even claim the map before you have any feedback
     
  20. Shogun

    Shogun L6: Sharp Member

    Messages:
    262
    Positive Ratings:
    228
    It might also be helpful to put a short overview of the feedback commands and what they do in the MOTD, so it's the first thing people see when they join the server.
     
    • Thanks Thanks x 1