Public Discussion: Testing and Random Crits

Discussion in 'Site Discussion' started by Fr0Z3nR, Sep 3, 2014.

  1. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,902
    Pandora's Box is being opened again.

    So I've had a few people who have been asking for a public discussion on Random Crits during our tests and other server events. I know this is a wonderfully hot topic for people, so lets please keep everything civil and respectful.

    Please note, we are talking about our TESTS, not necessarily our Beta Casuals or just Casual plays.
    Please also note, I am talking about Random Crits in the overall general sense (big macrocosm of things). You can talk about individual instances though on a microcosm of testing.

    Things to consider:
    • Random Crits are on by default for TF2 quickplay servers (though you can turn them off) do you feel like Random crits should be included in our TESTS because of this.
    • Do you feel that OVERALL, in the whole scope of a test, random crits severely sway a test?
    • One of the options we have currently is, if an AUTHOR wants a map tested without crits, they can request for them to be disabled. They are still tested WITH by defaul.
    • Do you feel that players dislike for random crits in tests is a personal preference they a trying to push onto the test, or because they genuinely feel that crits sway a maps play style overall?
     
  2. worMatty

    aa worMatty Repacking Evangelist

    Messages:
    1,071
    Positive Ratings:
    832
    This, basically. What's there to discuss? We're testing for the real world, and if not, then we're testing for a league, in which case the crits can be disabled. You can't have a fair test if you don't recreate the same circumstances that exist in the most common scenarios, or those that the map is intended for.
     
    • Thanks Thanks x 5
  3. Dr. Sasha

    Dr. Sasha L4: Comfortable Member

    Messages:
    184
    Positive Ratings:
    85
    Crits should be kept on. They're enabled on countless servers, without them the test gameplay would be unrealistic for more than 70 percent of tf2 pub servers.

    EDIT: Though, it can heavily affect gameplay, it's fine as long as we test the map multiple rounds.
     
    • Thanks Thanks x 2
    Last edited: Sep 3, 2014
  4. YM

    aa YM LVL100 YM

    Messages:
    7,099
    Positive Ratings:
    5,739
    Random crits are a way of leveling the playing field between those who are good and those who aren't good at TF2.

    In tests, we don't want any imbalance in teams, so we want a nice spread of player skill across the teams. We don't want a team with a pro level player (of which our community has several) dominating and ruining a test.

    Random crits allow a team that has poorer players to still get one up on the team with the pro players occasionally. It's not a whitewash, just a little boost.

    Also from my experience, it's only the players who are the highest skill level that complain about crits, and removing crits would only serve to boost their ability to dominate weaker players, leading to worse teambalancing in tests.

    IMHO it's important we have random crits.
     
    • Thanks Thanks x 4
  5. TMP

    aa TMP Abuser of Site Rules

    Messages:
    948
    Positive Ratings:
    551
    Crits or no crits doesn't really change the game at all when it comes to balance on a macroscopic level. So it really doesn't matter in the end.
     
  6. Stiffy360

    Stiffy360 L1: Registered

    Messages:
    20
    Positive Ratings:
    45
    IMO, since testing is mostly for seeing how well a map is balanced and random crits can destroy balance, I think its fair to disable the crits. Because if the base without crits is balanced, the map is balanced, and the upsets in balance would be skill sets and of course random crits, which the purpose of them is to destroy balance basically.

    That's untrue because the more damage you do the higher the chance of you getting a crit is. So the pro players or people who play demo and soldier get LOTS of crits while people who can't quite put out a lot of damage gets less. So it actually makes the balance of teams even worse.
     
    • Thanks Thanks x 2
  7. luxatile

    aa luxatile deer

    Messages:
    2,289
    Positive Ratings:
    1,348
    Just answering the questions raised in the OP for now:

    Yes, I do feel that random crits should be included in our tests because they are on by default. In my opinion it doesn't matter whether I think this is a good design choice or not; we are making maps for TF2, and not a for a modified version of TF2. Anything that happens in tests should be representative of how an average session on a map might go on a normal server.

    I don't think crits sway your overall test results in any way whatsoever. Yes, maybe you once or twice might get a victory in a test because somebody happened to kill half the other team with a crit rocket, but it's simply a harsh reality that sometimes unexpected things happen in video games. Unless we were to employ teams of competitive players for our tests, we would never be able to eliminate luck from the game, with or without crits. And in the big picture, once you do enough tests to make sure your data is reliable, luck will quickly cease to be a factor.

    I do, however, agree that the author, specifically, wants to turn off crits, it should be allowed. This is because I believe the author should be allowed some agency in how their maps are tested, and if they feel like it's important to their map that crits be off, then why not? TF2 does function perfectly well without crits, after all, it's just that I think it is generally the less good choice of the two.

    I don't think I have a properly developed opinion on whether or not people's complaints stem from personal preference, but I have a feeling this does happen (as evidenced by the fact that it's not exactly rare to hear somebody yell about crits being unfair over voice chat). I think it's unacceptable to make the decision to turn crits off if this decision is to be based primarily on people being upset with crits rather than something else - but as I said, I don't know what people actually think.
     
  8. Doktor Richter

    Doktor Richter L3: Member

    Messages:
    117
    Positive Ratings:
    78
    I still don't know exactly how I feel about random crits, for testing and for TF2 in general.

    On the one hand, I feel like they're technically fair, since both teams have access to them. Since you can increase your crit chance through damage, they do technically reward high-level play, just like raw skill does. Plus, many weapons are balanced by having "No Random Crits" as a downside. Technically, disabling random crits is buffing these weapons ever-so-slightly.

    On the other hand, crits completely take away the player's ability to control their fate in battle, which goes totally against basic game design. And, a random crit could instantly finish a battle that would have otherwise given the map author valuable information; in other words, you can't gain much useful feedback from a battle if one player turned a corner and was shredded instantly by a crocket. It goes without saying that, in our short 20-30 minute tests, authors want as much useful information as possible. And one could argue that the suite of weapons in TF2 isn't very well balanced anyway, so the "No random crits" downside is sort of moot. And personally, I feel like I can't improve my skill/map knowledge when killed by crits, and I feel like a buttmunch when I kill someone with a crit.

    Should random crits be on/off depending on whether the author wants to make a quickplay map vs a pro/comp map? Do fights ended by random crits give any useful data for map authors? Do such fights prevent useful data from being generated by what would otherwise be fair fights? Will the balance of weapons be shifted significantly by disabling random crits? I think these are some things we want to consider when choosing to test with/without random crits.
     
    Last edited: Sep 3, 2014
  9. ParanoidDrone

    ParanoidDrone L3: Member

    Messages:
    147
    Positive Ratings:
    10
    Over a sufficiently large sample size, the law of averages should come into play and normalize the effect of crits on the grounds that both sides will benefit equally from them. In theory. The question then becomes whether or not the average map undergoes enough tests to reach this point. I don't have the answer to that, maybe someone else does.

    I can see arguments going both ways. A map designed for competitive TF2 will of course assume nocrit and its tests should reflect that, but there's nothing stopping a pub player from finding the map and introducing it to their crit-enabled server. Leaving crits enabled for testing can expose areas where a lucky crit (or a Kritzkrieg) can wipe out a large portion of the enemy team, but is that the crit's fault or is it just exaggerating the symptoms of bad map design?

    The thought occurs to me that nocrit exists to standardize damage output by removing the outrageous outliers of crits. Therefore one could argue that it's best for tests to be nocrit since the results would also be standardized with few outliers. But when a map is released into the wild, where crit-enabled servers exist, your neat standardized tests are no longer quite so valid. It's an interesting puzzle.

    In the abstract I would say "cater to your audience." Nocrit tests for competitive-designed maps, crits for pub-designed ones. The problem is that any given map can be played by either group.

    (Christ this post has nothing resembling a coherent line of thought. Sorry.)
     
    • Thanks Thanks x 2
  10. Statboy

    Statboy L1: Registered

    Messages:
    1
    Positive Ratings:
    0
    I don't think there are many random newbs that find their way into one the tests, so we don't really need random kritz to balance them out with the comp players
     
  11. Turbo Lover

    aa Turbo Lover Fight me under Glasgow Central Station

    Messages:
    333
    Positive Ratings:
    317
    The crits are real.

    I don't like crits as a personal preference, but I think they're especially inappropriate for out map playtests.

    The big argument for keeping them during tests is because that's the setting Valve's pub servers use and that's the setting most players experience a map with. Firstly, Arena; Arena is not hosted on any Valve pub servers, the only precedent we have for Arena configurations is taken from the NA server hosts [arenafortress] - which turn crits off, but that's mostly because crits are extra bullshit in Arena (And Medieval). There is no standard configuration for Arena, it's just convenient for us to copy what other people are doing with their configs.

    Ignoring this edge case however, I put no stock in the argument that we should enable crits "Because Valve do it" I think it's indicative of a community of sheep who can't think for themselves about what is best for our tests, our server, and our community. But beyond that, we need to stop thinking about the server configurations as being the best for public play (As determined by Valve) because that's not what we're looking for, we want the best configurations for public testing; playing and testing are very different and should not be treated identically by our server configurations.

    The general consensus of the TF2 community is that crits are fun that's why most servers have them, but it's not the kind of fun we should be considering. Layl undoubtedly finds setting up level three sentries by our spawn doors in Snowplow fun; Berry undoubtedly finds playing Caber Demo with the Sticky Jumper and B.A.S.E Jumper fun, and I'm sure if I am dominating somebody on the enemy team and they get their revenge kill with a crit rocket, they find it fun to deal with me. Do any of these things count as feedback? No. Are they useful for determining what should be changed in the next iteration of the map? No. Should we facilitate these things if they aren't doing anything for our tests? No.

    Ultimately that's what it boils down to, instead of asking "Do I like crits?" or "Will the public be playing my map with crits?" we should be asking "What useful information can be gleaned from enabling crits?"

    And I don't think we can find out anything useful by enabling them. Ask any self-respecting engineer, software or otherwise, but if you ask him "Should we introduce a random element to our testing procedure?" and he'd probably slap you right then and there. And that's what we need to think like, engineers, we need to be analytical and rational in our approach to map testing. And random crits are anything but.
     
    • Thanks Thanks x 2
    Last edited: Sep 3, 2014
  12. A Boojum Snark

    aa A Boojum Snark Toraipoddodezain Mazahabado

    Messages:
    4,767
    Positive Ratings:
    5,508
    Basically that. RNG crits can change the outcome of A round. RNG crits can not change the outcome of many rounds collectively.

    If someone has too few tests to rule out crit factor, they have too few tests to rule out skill imbalance and how well a player feels on the day they are playing.
     
    • Thanks Thanks x 5
  13. Fr0Z3nR

    aa Fr0Z3nR Creator of blackholes & memes. Destroyer of forums

    Messages:
    6,391
    Positive Ratings:
    4,902
    As a scientist, with a degree in science and who has done something like this before:

    If the environment has random variables, yes, we want them. If thats the environment, it's worse if we don't have them. It's called statistics, we can filter them out via math.

    Good stuff so far guys.
     
  14. Berry

    aa Berry Sharing is caring, and I don't care

    Messages:
    979
    Positive Ratings:
    1,461
    I've only browsed down the thread, and my opinion probably shows bias (as expected).

    I believe it should be down to the test host to say "do we want random crits?" to everyone and take it from there. Of course currently there's no plugins for stuff like this to be modified without rcon.

    I think both sides are correct. Random crits are a part of the game - and we're testing the entire game really - but random crits also don't offer much to testing on a personal level for both the players and the mappers.

    Let me ask you this; what can you get from "blue pushes forwards to C, red holds them off until X has a random crit, blue caps the point". Nothing.

    It doesn't tell you if the map's C point works, it doesn't tell you if red is having trouble defending. It tells you someone got a random crit because of an annoying mechanic.

    Onto the side of the testers, random crits feel annoying to be killed by - and to some people annoying to kill with - and really if people are being annoyed by something like that is it really worth keeping?

    It's the same case as CTF cap crits. It's part of the game, yeah, but it's also an absolutely stupid mechanic.


    My personal suggestion for a solution:
    - "!sausage" style vote to turn off/on random crits.
    - Command for donators to send a random crit vote

    Overall, the random crit debate has been going on for years. It won't end because both sides have a balanced arguement. Really I think the only solution is a "suit the majority" method.

    This is just my two cents though.
     
    • Thanks Thanks x 2
  15. LeSwordfish

    aa LeSwordfish semi-trained quasi-professional

    Messages:
    4,102
    Positive Ratings:
    5,990
    I agree with Berry, essentially. put a voting system in that we can stop worrying about and let people get on with it.
     
  16. puxorb

    aa puxorb

    Messages:
    436
    Positive Ratings:
    576
    After reading through most of these posts I feel that I should share what I think as well.

    As TurboLover said adding crits adds nothing to the testing of a map at all.
    However, as many people have already said and can agree with, removing crits likewise adds nothing significant to testing.

    So I think it should be up to the map-maker whether they want crits enabled or not. Just keep in mind that if you do disable them, I might not play on your map. I usually leave servers that have random crits disabled because for me tf2 is supposed to be fun and chaotic; unpredictable.

    So does fun matter for map testing? Not really, but don't expect me to be there if you remove crits.





    Edit: (Also I use my numbers to switch weapons because my scroll wheel is super sensitive. So the less votes that I am accidentally going to choose an option on, the better.)
     
    Last edited: Sep 3, 2014
  17. A Boojum Snark

    aa A Boojum Snark Toraipoddodezain Mazahabado

    Messages:
    4,767
    Positive Ratings:
    5,508
    You really think that's going to work? I only forsee it causing more complaints. If something just is how it is (crits on or off), people will get used to it or just accept it. If every event starts with a vote to decide it, you will have people unhappy that the vote didn't go their way. Do you want to see "that wouldn't have happened if you guys didn't vote that way" over and over? Or perhaps people joining after the vote and going "oh crits are X, bye guys" ...I don't think a vote is the best option at all!

    edit: hey look at that, puxorb got his post in before me and is already confirming my point.
     
    • Thanks Thanks x 1
  18. theharribokid

    aa theharribokid

    Messages:
    867
    Positive Ratings:
    806
    I'm all for random crits on our and all servers but I think it should be pointed out for people that don't know. Random crits are not perfectly random, they are also based on recent performance, damage dealt over the last 20 seconds, check this video out for a more specific explanation https://www.youtube.com/watch?v=HHEMS68C5DY

    EDIT: I am definitely not for crit voting while on the server, I find the map voting annoying enough, I wish we could change that so we only get asked to vote once we've died or something but that's another point
     
  19. Sergis

    aa Sergis L666: ])oo]v[

    Messages:
    1,871
    Positive Ratings:
    1,130
    testing wise i find their impact negligible. over a few years i can remember only one case when a crit changed the match significantly

    personal wise i find no random crits just dull and dont care to play without them
     
    • Thanks Thanks x 1
  20. RubbishyUser

    RubbishyUser L7: Fancy Member

    Messages:
    414
    Positive Ratings:
    242
    Let me preface this by saying that I am in favour of random crits on the majority of weapons, but:

    This is actually the best reason for turning off crits. See, one round is nothing in the extensive testing afforded to Snowplow, the Valve Beta maps and internal Valve testing, but many maps that we play often only get 1 or two tests before a new iteration is made. In some game modes, that means we only get 4 rounds in. A perfect crocket could then mean Blu wins an extra 25% of the rounds, and the mapper changes the map accordingly.

    This might only happen 1 in 50 rounds, but I'd hate to think people are making a step in the wrong direction over a single rocket. Anyone who wants to see the effect of random crits, load up goldrush with some bots and spectate and see how much a crit grenade can do.