Hello, Dr. Krane. Long-time poster, first-time... errrrrr... dancer?
I read all the comments about the quality of feedback, that people submitted via the survey, and made some notes.
Some people feel like the convenience of the feedback plugin discourages larger and more detailed feedback from being posted on the forum. Maybe this is true, it's certainly more convenient to leave a brief line of text during the game than it is to recall one's thought and write an essay later on, but the character limit of TF2's text chat does encourage one to be concise, which is also beneficial. In my experience, fellow mappers tend to reserve forum posting for times when they have a lot to say, or feel that the map or mapper could genuinely and greatly benefit from in-depth analysis. The process of making personal notes during play testing, perhaps giving a map a run-around locally for a few minutes, and finding the map's forum thread to post paragraphs, diagrams and screenshots, is tedious and inconvenient. It takes a good chunk of someone's valuable time. Perhaps we could make this process easier with enhanced functionality of the feedback system.
For example, it could be possible to introduce a new command that allows a player to make private notes during a play test. When the map is done, they could be sent to the player's forum inbox. The player could then use these as prompts to post more detailed feedback in the map's thread, along with any screenshots they happened to take at the time. The notes wouldn't be available to the mapper. SourceMod chat commands can be issued silently when the exclamation mark ( ! ) is replaced by a forward slash ( / ).
On the other hand, some survey users seem to feel that, as map authors, it's inconvenient to chase up people who left feedback that they felt was too brief, for help in understanding the problem and for ways to deal with it. When I've needed clarity on some feedback, I've sought the submitter, but I was only able to do this because I knew them, or knew how to track them down on Steam, but it took a long time to do this for all the feedback I had that needed investigating. The map feedback pages could be modified to allow authors to copy and paste blocks of feedback in to a forum post, along with the forum username of each feedback submitter, prefixed by an '@'. The submitters would then be notified by the forum that they had been mentioned in a thread and could respond to the request to elaborate. This change could have a knock-on effect, causing map authors to make more use of their map threads, involving feedback submitters more, and bringing discussion about feedback, layouts, gameplay and so on to the fore, where less experienced people will read it and learn about best practices.
Another common issue that was highlighted was the value of small pieces of feedback that seemingly aren't well-thought-out, or of superficial things. I think this speaks more about how map authors interpret and use feedback than it does the people who leave it. Anyone who is new to the site, has bravely made an account and has ventured in to a play testing session is probably scared they might say something wrong, but they really want to help. They may not have the knowledge to explain why they feel something, or might be so excited about the opportunity to get involved with play testing that they look for small things that they can comment on safely. The fact is, these people want to help, and they should be encouraged. You don't have to take any of it seriously, but if you spend time trying to understand the submitter's motives for a particular piece of feedback you may find that significant unspoken problems are revealed to you that would otherwise have been collectively ignored. If you stick with these people, ask them open questions and nurture them, you may find they stick with you, and you get the benefits of an enthusiastic play tester whose future experience and perception is very valuable.
All kinds of feedback should be continuously encouraged, even the seemingly negative stuff, so that submitters feel free to keep doing it. Comments about small bugs, glitches or cosmetic things in an alpha map can lead to useful discussions that could spawn new ideas. Play testers could be shown a guide on what to look out for, and how to construct good feedback. This could be summoned using a special chat command, or shown on the MOTD page. It should be required reading for new forum users, or anyone who wants to get in on play testing but doesn't yet have enough experience or knowledge to produce good feedback. Something brief, with examples, pictures, that exposes the reader to the existence of commonly-understood areas of design, such as: layout construction that accommodates class-specialised gameplay, intuitive navigation to and awareness of objectives, readability and visual comfort, the ebb and flow of battle (battle intensity that rises and falls because of timing or space. Knowing when to have periods of rest), and of course fun.
To help alleviate concerns about feedback that is not well-thought-out, a modified version of the Game Day testing format may increase the value of feedback during an impromptu play testing session: The first half of a map's duration is played normally, with sv_alltalk disabled (voice communication would only be possible with one's team) and text feedback discouraged whilst a player is alive. The second half of a map's duration would see sv_alltalk switched back on, along with no restriction on feedback or discussion. This could help a map author by giving them play testing data where the participants are less distracted, and are more focused on the game than looking for problems. A step further would be to encourage the use of a forward slash with all text feedback so that it doesn't show up in chat, and therefore doesn't influence or compromise any other player's perception of a map. Though there are pros and cons to hiding and showing feedback.
I look forward to more hypotheses! Is that right?