Public Discussion: Testing and Random Crits

[Bakscratch]

On the topic of data purity and "a map iteration only gets 1-3 tests," I don't see any reason why people can't have a single version of their map tested multiple times. -snip cuz-

I did this with Ripley only to get people saying, "we have played this version before" then getting RTV'ed.

Btw Crits are part of TF2 its a luck based system, same thing in dota heroes with RNG, are "OP" only a % of the time as. But crits in TF2, people only get mad when they get hit by one, (Which they probably would of died from even if was a normal shot) If we are talking about comp then, that should be for team of players not random people on a server.

Crits do not effect a test at all, hell it even helps as it give a more realistic test as the majority of servers have crits on, so turning them off is just pointless.
Plus the odd crit now and then isn't enough to break a map, they are crits they don't change enough to do anything.
Its all down to players getting mad about getting hit from one, even though they were probably gonna die. Its just another thing people can complain about honestly.

 

[wareya]

crits affect data from small sets of tests negatively
crits affect data from large sets of tests positively
science and statistics say so
can we please stop arguing about crits and talk about interesting things please????

 

[Bakscratch]

crits affect data from small sets of tests negatively
crits affect data from large sets of tests positively
science and statistics say so
can we please stop arguing about crits and talk about interesting things please????

Speaking how 95% of test people complain about them its a problem. Its either just the player or the game, and im going with its not the game.

 

[TMP]

crits affect data from small sets of tests negatively
crits affect data from large sets of tests positively
science and statistics say so
can we please stop arguing about crits and talk about interesting things please????

Ok, so here's something cool.

Assuming we map the effect of crits on a test as a function f(x) where x is the number of tests.

If we have that f(x)<0 for x<M lower than some M in Z, and f(x)>0 for x>N at some point for some N in Z, then if we approximate this discrete function on Z+ by a continuous function on R+, then mathematically, by the mean value theorem, there should be some value of testing such that the effect of whether or not crits are on or not is 0.

So we should find that number and test every map at that number. PROBLEM SOLVED, WE CAN'T COMPLAIN LOGICALLY UNDER THAT CIRCUMSTANCE.

We've got some maps with large amounts of gameplay testing, so let's try to find that number!

 

[wareya]

@ bak:
it's not as simple as it's the player or the game, there's other shit than the player and the game when you're testing. there's the server's playerbase (in tf2m's case, scattershot-quality pubs) and the server's momentary purpose (95% of the time, small scale tests), both of which in the case of impromptus mean that crits are mostly harmful.

im saying we should stop arguing about the effects that randomness have on the usefulness of a data set (feedback, heatmaps, etc) because they are a given. we should talk about anything else like whether crits are harmful in some other way like how we know they alter damage balance, same as weapon spread (huntsman 1-shotting scouts, rifle not 1-shotting medics, changing minigun DPS slightly depending on range).


@ tmp:
i did not respond to this post but this section is here to prevent confusion

 

[wareya]

double post to prevent confusion again sorry!!!

to respond directly to the OP:

>Random Crits are on by default for TF2 quickplay servers (though you can turn them off) do you feel like Random crits should be included in our TESTS because of this.

considering that 10% (and growing -- or rather, not shrinking like the rest) of the tf2 playerbase plays without random crits (or spread), i feel like whether they're normal or not is not relevant.

>Do you feel that OVERALL, in the whole scope of a test, random crits severely sway a test?

yes. a majority of tests have significantly insignificant sample sizes and would generate better data without randomness.

>One of the options we have currently is, if an AUTHOR wants a map tested without crits, they can request for them to be disabled. They are still tested WITH by defaul.

most authors aren't familiar with arcane things like community policy or social interaction. i feel like crits should be disabled by default because of their potential to harm most testing data sets.

>Do you feel that players dislike for random crits in tests is a personal preference they a trying to push onto the test, or because they genuinely feel that crits sway a maps play style overall?

it's both. to be frank, whether or not it is a personal preference is not important; politics is about opinions, and if you're here to ask us about this kind of stuff, you'd better be ready for subjective things. you can figure out the science parts on your own if you let yourself go lucid enough, you're a smart guy.

 

[Fruity Snacks]

Ok, so here's something cool.

Assuming we map the effect of crits on a test as a function f(x) where x is the number of tests.

If we have that f(x)<0 for x<M lower than some M in Z, and f(x)>0 for x>N at some point for some N in Z, then if we approximate this discrete function on Z+ by a continuous function on R+, then mathematically, by the mean value theorem, there should be some value of testing such that the effect of whether or not crits are on or not is 0.

So we should find that number and test every map at that number. PROBLEM SOLVED, WE CAN'T COMPLAIN LOGICALLY UNDER THAT CIRCUMSTANCE.

We've got some maps with large amounts of gameplay testing, so let's try to find that number!

Is it bad that I actually almost thought about solving this problem?

Anyways, 5 pages in and the thread has basically devolved into what I fully expected to happen. Some people giving one reason (fully logical), other people leave another reason (still, fully logical) and no one budging on anything. We have the science reasons (see My, Rubbishy and TMP's post), we have the other reasons (see everyone else). We have personal opinions of RNG/Non-RNG. We have so many reasons and no one has made any good clear heads or tails about it.

I remind everyone to keep the arguement CIVIL and RESPECTFUL. I will NOT hesitate to shut the thread down and squash ANY discussion about it. If people want things to change, you will need to follow that rule.

 

[RubbishyUser]

Ok, so here's something cool.

Assuming we map the effect of crits on a test as a function f(x) where x is the number of tests.

If we have that f(x)<0 for x<M lower than some M in Z, and f(x)>0 for x>N at some point for some N in Z, then if we approximate this discrete function on Z+ by a continuous function on R+, then mathematically, by the mean value theorem, there should be some value of testing such that the effect of whether or not crits are on or not is 0.

So we should find that number and test every map at that number. PROBLEM SOLVED, WE CAN'T COMPLAIN LOGICALLY UNDER THAT CIRCUMSTANCE.

We've got some maps with large amounts of gameplay testing, so let's try to find that number!

OK, TMP is letting some of his math show (what do you/did you study?) but I think I understand what he's talking about, and it could actually really matter on a larger scale if we can work it out. TL;DR version is that we should be able to use statistics to find the playtesting time that results in the maximum gain for the minimal amount of time wasted.

I'm too tired now, but I might have a squint at this again tomorrow. We have the data now: or rather, Geit does.

 

[wareya]

he means we should play for exactly the "duration" at which statistics say that this environmental randomness doesn't affect the "goodness" of the data

 

[MystycCheez]

A golf course's green is the most up kept and maintained part of a golf course because any imperfections can change the direction of the ball and the outcome of a game. If a stray blade of grass is too long and a ball rolls over it, the ball may go another direction. A green in poor shape is like random crits, sure it might make the game more interesting to see who gets the lucky break but in the long run it just eliminates skill and ruins fun.

 

[Fruity Snacks]

he means we should play for exactly the "duration" at which statistics say that this environmental randomness doesn't affect the "goodness" of the data

Which is either 0 or a long time.

 

[Prestige]

Crits should have been balanced in TF2 itself a long time ago. Random 300% damage, "rolling" crits, and 300% explosive radius is far too much RNG. Then the kritzkreig came out and weapons with crits stored and what not so maybe it's too confusing to change them now.

Crits have been a divisive issue on TF2M for centuries now, we might as well put up an annoying vote at the beginning of maps until people get sick of it and give in to the other side.

 

[wareya]

IMO crits should've been removed from TF2 when alternate weapons happened. They give better ways to feel useful as a bad player getting stomped. This is a whole separate topic, though.

 

[Ælement]

Random crits adds nothing but inconsistency to tests. And even there, its effects on the gameplay are pretty much negligible.

Pair that up with the fact that random crits makes your deaths feel undeserved and your kills feel cheap and unsatisfying, and i'd say we'd have plenty of reasons to get rid of that brainfart that random crits are.

 

[Another Bad Pun]

crits are fair and balanced

/thread

 

[Sergis]

crits are fair and balanced

/thread

and real, dont forget real

 

[Turbo Lover]

If both sides stubbornly (and rightfully) believe they're correct, shouldn't we save everyone time and mental exhaustion by trying to work out a compromise now rather than wait for another handful of pages that contain nothing new?

We're kind of limited in what kind of compromises we can come up with, but this is a very old topic, it'd make sense if no new arguments cropped up. But I don't think that we can change nothing about our testing procedure after this. The arguments for and against are too balanced for either to be applied to our current testing setup because someone says so.

 

[Fruity Snacks]

If both sides stubbornly (and rightfully) believe they're correct, shouldn't we save everyone time and mental exhaustion by trying to work out a compromise now rather than wait for another handful of pages that contain nothing new?

We're kind of limited in what kind of compromises we can come up with, but this is a very old topic, it'd make sense if no new arguments cropped up. But I don't think that we can change nothing about our testing procedure after this. The arguments for and against are too balanced for either to be applied to our current testing setup because someone says so.

I think leaving it the author to decide if they want crits or no crits is the best option. I trust everyone to not complain and be respectful of the authors decision during gamedays, impromptus and whatnot.

Actually EDIT:
The one thing I'm seeing in terms of the arguement to turn Random Crits off during tests is that overall in a small sample size, they have the potential to have a large effect on the outcome. I do not disagree with this, is is a scientific and mathmatical truth. But, I think the arguement presents a bigger issue within the community. We rush. I see it quite often where someone will test a version of a map for one, 3 round gameday test and then make big changes based on that. There is no race to finish the map, there is no reason that the same version of a map shouldn't be tested 3-4 times over the course of a couple weeks, at least, on paper. We have rules that allow this type of thing, and if there are rules that prohibit the testing of multiple version, then they should be changed.

What I do see though, are authors who try and get the same version of the map tested multiple times to get more feedback. Aly's been trying to do it with Inari, she's had the same version of the map tested a lot, but hasn't recieved enough feedback to comfortably move forward with changes. I've seen other authors (including myself) put the same version of maps into tests 3, 4, sometimes 5 times during the span of a week or 2, and after only the second time being played, it gets RTV'd "because we've played it already." TF2Maps.net is a community of map makers and map testers. First and foremost, we're a service to the TF2 community at large as a place to test all maps, new, old, innovative or just trying to do the usual thing again. I have said this in the past, and I continue to bring it up now. We're starting to lose touch with our roots and it's becoming clear that the true purpose of TF2Maps.net is no longer seen by some members of the community.

We're a map testing community, first and foremost. A service and community to the TF2 community at large.

 

[wareya]

That already exists, as long as the person running the imp is cooperative. Isn't this all about what the default should be?

 

[Turbo Lover]

I think leaving it the author to decide if they want crits or no crits is the best option. I trust everyone to not complain and be respectful of the authors decision.

Which doesn't cover maps where the author says nothing about the crits, which is currently the vast majority of our maps. And once the map has changed, would we continue to use the setting of the previous map in the absence of author preference or return to the default setting, whatever that may be.

I agree that the map author's decision is final, but only a few authors actually give their decision while most just run with whatever the server has on. Do we ask every author their crits preference when we're organising the test, "Map name, download and crit preference, please."? And even then, what if they don't care and have no preference.

What you said is correct, but it isn't a solution.