Public Discussion: Testing and Random Crits

Sel

Banned
Feb 18, 2009
1,239
2,570
It got an infraction because this is not the thread for discussing what tf2m is supposed to be; if the post only consisted of the first paragraph I don't think it'd've gotten an infraction

Except that's the real issue here, whether anyone wants to accept that or not, and even if it wasn't, I'm pretty sure there's a huge difference between "Flaming/Trolling", and "Off topic".
 

Sergis

L666: ])oo]v[
aa
Jul 22, 2009
1,874
1,257
Except that's the real issue here, whether anyone wants to accept that or not, and even if it wasn't, I'm pretty sure there's a huge difference between "Flaming/Trolling", and "Off topic".

as big as the difference between those two things might be you're pretty good at combining them
 

Pocket

Half a Lambert is better than one.
aa
Nov 14, 2009
4,694
2,579
This thread is pathetic. There is no argument on whether crits should be on or off in a testing environment. Anyone capable of basic reasoning can put together that if your goal is to create an incredible, well balanced map you'd have to be phenomenally stupid to introduce a diceroll that massively effects the outcome of a tf2 match. It really is that simple.
It really isn't, when that diceroll is also present on the vast majority of the public servers that would, hopefully, eventually be playing the maps that get enough attention and acclaim to go official. Like I said, in theory testing without crits will get more accurate results for both environments than testing with them, but we can't very well prove that if nobody will ever let us test without them except for a handful of maps whose creators remember specifically say to every single time their map comes up in rotation.

The problem is that nobody seems to care one way or another about that, and treats this issue like it's purely a matter of personal preference. You and I seem to be the only exceptions, and frankly you're doing more harm to my side than good by being on it.
 

Fruity Snacks

Creator of blackholes & memes. Destroyer of forums
aa
Sep 5, 2010
6,394
5,571
It really isn't, when that diceroll is also present on the vast majority of the public servers that would, hopefully, eventually be playing the maps that get enough attention and acclaim to go official. Like I said, in theory testing without crits will get more accurate results for both environments than testing with them, but we can't very well prove that if nobody will ever let us test without them except for a handful of maps whose creators remember specifically say to every single time their map comes up in rotation.

The problem is that nobody seems to care one way or another about that, and treats this issue like it's purely a matter of personal preference. You and I seem to be the only exceptions, and frankly you're doing more harm to my side than good by being on it.

I've chatted with some people for setting up an actual experiment to test this. Spend a few weeks without crits, then a few weeks with. See what happens. But the controls needed and the amount of attention needed to properly do such a test is kind of high. Resources (management, recording, etc) would be relatively expensive.

I will still stay with, statistically speaking, if a map is played for a few hours over multiple tests then the random crits have a significantly lesser impact on the overall results, than if it is just tested with ONE 30 minute round. The community needs to understand that we're not just "a community that plays custom maps", we're one that "tests" custom maps. As such, playing the same version of map over and over, if the author wants it, is necessary. Especially if Random Crits are enabled.

I have suggested to the staff that we should do a vote, just to get an idea of the numbers in each camp. Nothing to decide anything, yet, just to see if the anti-crits camp is a small vocal majority or not.
 

Egan

aa
Feb 14, 2010
1,375
1,721
Some observations overall from the thread:

In early versions of your map, when glaring flaws present themselves, typically you don't want to test your map versions repeatedly; you'd prefer to create new versions relatively fast until your map becomes stable. Early map tests typically span around 30 minutes, and during this time you want the most accurate data, and to get the most accurate data you want the lowest amount of randomness available - no random crits.

When your map becomes stable enough to handle multiple map tests (alpha 5 ish), the choice between having crits on or off comes down to the author's preference. If you're making a map for an environment where crits will be off (competitive maps) then you should have crits off for these stable-map map tests. If you don't care where it ends up, it's just your user preference - do you like crits or not.

When your map becomes even more stable, enough to be moved around to other server environments (beta usually), then testing with crits on and off is equally as important, because you want the most amount of environment variables tested (crits on or off, player skill, etc).

Early map tests: Crits off.
Stable map tests: Preference.
Stable enough to move to other servers: Both.

Having a vote may be useful for introducing the high-level opinions from people who don't want to read through the posts.
 

worMatty

Repacking Evangelist
aa
Jul 22, 2014
1,257
999
I don't think something that has a 2% chance to happen will really have an important effect on the outcome of a set of test results.

Really, player skill is a much wider variable than crits. It affects a game much more often. But we aren't arguing about that are we?

I think that the majority of useful feedback and play testing is returned throughout the whole of a game. I don't think that an author would need to know that a round was won because of a single crit. He would want to know whether the map felt fair and fun to play.
 

Crash

func_nerd
aa
Mar 1, 2010
3,315
5,499
I don't think something that has a 2% chance to happen will really have an important effect on the outcome of a set of test results.

Really, player skill is a much wider variable than crits. It affects a game much more often. But we aren't arguing about that are we?

I think that the majority of useful feedback and play testing is returned throughout the whole of a game. I don't think that an author would need to know that a round was won because of a single crit. He would want to know whether the map felt fair and fun to play.

Exactly. We have everyone from upper tier competitive players to people still rather new to the game testing on our server at times. I'm not sure why crits being a variable is that big of an issue when the rest of the data is so all over the place.
 

fubarFX

The "raw" in "nodraw"
aa
Jun 1, 2009
1,720
1,978
I think this debate is dumb and that this is really the least of our concerns when it comes down to the quality of tests but here's something I can contribute to this discussion.

Sometimes, one crit is all it takes to break a bad choke. Sure, one crit can tip the balance for one round but observing the effects of crits on the win percentage of rounds is pretty dumb as crits are mostly meaningless in the big scheme of things. However, the thing to note is that having crits off makes things a lot less forgiving for your map as you can't count on the occasional crit to alleviate inherent issues with your map. (like bad chokes or questionable height advantages)

so I guess my point is this, if you can make your map perform well without crits (which I identify as a more rigourous test environment) it will most certainly perform well with crits on as well. so why not cover all scenarios?

1. turn off crits
2. make sure your map performs well in all circumstances
3. ???
4. profit!

(don't think that I care one bit tho, because I really don't. But I do care about logic and it stands to logic that turning crits off is the better testing condition)
 

A Boojum Snark

Toraipoddodezain Mazahabado
aa
Nov 2, 2007
4,775
7,669
I value the logic of your argument and support it, except that the small difference in the lack of crit rockets or minigun sprees also shows up in things like melee weapons that are designed to crit often and reliably, with alternates whose downside is no crits. Not having a crit spree to get through a bad choke point might be offset by the fact a spy can now take out the sentry on the other side of it easier because the engineer isn't going to critwrench (high base crit + performance boost because his sentry is killing stuff in the choke) him in the face while removing the sapper.

I don't not care either, but I don't think this discussion has ever been about what is better, it is about what is less worse.
 

Seba

DR. BIG FUCKER, PHD
aa
Jun 9, 2009
2,364
2,728
It is obvious the only solution is to use a sourcemod that enables crits only for melee weapons
 
Sep 7, 2012
638
500
Can we at least do the simple config file asap? That way we're still able to use a point_servercommand without changing any other testing environment's preferences.
 

Pocket

Half a Lambert is better than one.
aa
Nov 14, 2009
4,694
2,579
There are also non-melee weapons whose downside is no crits. Valve should have made it so that if you're on a server with crits off, those weapons have some other downside like less base damage or something.