Stamps statistics falsification

Discussion in 'Team Fortress 2 Talk' started by Geit, Dec 31, 2012.

  1. Geit

    aa Geit 💜 I probably broke it 💜

    Messages:
    597
    Positive Ratings:
    1,112
    Stamp statistics falsification

    Those who've been around TF2M for a while will know that I have an uncanny love of data, and of collecting it. Most of you will probably remember the stamp tracker that I tacked onto the feedback system about a year and a half ago and kept running for about 6 months, until I got too many data points to feasibly plot with the flash based graphing library I was using (160k data points over 6 months).

    Anyway, I made a post a few days ago in the news thread that was cross posted from the TF2 Blog announcing the revamped rewards for Map Stamps and the addition of map-specific strange filters. Just in case you missed it, here's the chart of map stamps that I produced:

    [​IMG]

    As you can see: most of the maps have a huge amount of stamps behind them, which would easily put their authors in line for some fairly large payouts. The strange thing being, no map author has even had anywhere close to even the most conservative estimates I could come up with.

    Firstly, I should explain where the data for this graph, and the original tracking page comes from: Valve provide a series of leaderboards which typically show only the top 225 stamp owners, however the leaderboard system in steam also has an XML API (which can be accessed by appending ?xml=1 to the end of the URL) attached to it that can be used to get the full list of stamp owners, and the amount of stamps they each have. When I started tapping into this data source, I had no reason to doubt it's legitimacy, it should be based on Valve's own internal data and be accurate down to the last stamp. Apparently not, I can now say without doubt that the data source has been falsified.

    After having talked to Rexy, who has been a great help in tracking this down over the past day or so, I decided to search around for my old data archive and trawl through it and look for any evidence of falsification, I honestly wasn't expecting to find anything, but as I was trawling through the rows of data I noticed this period of sudden growth:

    [​IMG]

    Each subdivision on this graph represents one hour of time, it shows the growth rate for Mann Manor over a few days. On either side of this dataset the average growth rate was about 2 per hour, on this part of the dataset it fluctuates between 500 and 14000 an hour.

    The period of growth shown above accounts for 80% of all map stamps recorded for Mann Manor.


    This same period of growth occurred for all other maps at the exact same time, in total it would have required around $1.25 million USD in legitimate sales to generate. Yeah, that's definitely not right. I'm not sure if the data has been falsified because of a bug within Steam, a rouge hacker or possibly even by Valve for some reason, it's all very odd either way. (Note: The amount of people who actually had stamps, according to Steam, also increased similarly during this period)

    I've taken the liberty of calculating a new estimate by removing the estimated inflation from each map's total count, for those that are curious:

    [​IMG]

    I've checked these statistics with Rexy, and he's verified with me that they are almost bang on the money in terms of the (estimated) revenue he's earned from map stamps on Mann Manor.

    I'll be reaching out to Valve for comment on the matter, and will be sure to update you if I get anything back.

    Have a happy New Year!
     
    • Thanks Thanks x 10
    Last edited: Dec 31, 2012
  2. Trotim

    aa Trotim

    Messages:
    1,180
    Positive Ratings:
    995
    Yeah, these seem to make a lot more sense based on what I know about the Coldfront stamps (if they're not still on the high side).
    Thanks for clarifying this a bit more
     
  3. A Boojum Snark

    aa A Boojum Snark Toraipoddodezain Mazahabado

    Messages:
    4,769
    Positive Ratings:
    5,527
    Valve should give you a unique tinfoil hat.
     
    • Thanks Thanks x 3
  4. WolfKit

    WolfKit L3: Member

    Messages:
    128
    Positive Ratings:
    51
    It costs less to buy a stamp pack than it does to buy the stamps individually, right? Assuming that map makers also get paid less when someone buys a stamp pack but it still counts as full stamps, that would account for getting less money than calculated with the amount of stamps and the full price of a stamp.
     
  5. Geit

    aa Geit 💜 I probably broke it 💜

    Messages:
    597
    Positive Ratings:
    1,112
    The estimated payout for a single stamp is 75 cents, the estimated payout for a stamp pack is about 50 cents, when I calculate revenue I calculate the highest possible revenue (Stamps * 0.75), the modal revenue (Stamps * 0.64) and the lowest estimated revenue (Stamps * 0.52) and plot them all on the same chart. It's a fairly safe bet that the actual revenue lies between the Modal and Low values for most maps.

    Prior to me finding this artificial growth in the data, these estimates frequently came out above 50 thousand US dollars, which is far more than any map author has actually gotten. The new estimates are almost pretty much correct to within about $750 USD.

    EDIT: I should point out that there are also various other factors at work when doing these estimations, such as disparities in the value of stamps across currencies.
     
  6. Micnax

    aa Micnax I maek map

    Messages:
    2,056
    Positive Ratings:
    1,382
    Good to see this being cleared up, now it actually makes Gullywash and Kong King look appreciated instead of being left behind by a few 40,000 stamps :p
     
  7. Geit

    aa Geit 💜 I probably broke it 💜

    Messages:
    597
    Positive Ratings:
    1,112
    I'm still watching the numbers, waiting for a sudden drop, so I'll update again if and when this happens.
     
    • Thanks Thanks x 4
    Last edited: Jan 8, 2013
  8. Janeator

    Janeator L2: Junior Member

    Messages:
    66
    Positive Ratings:
    2
    Ah , now this clarifyes everything. They look much more natural to me now. Except for 1 thing (which i guess it is still natural, sight): Why is Turbine the top 1 map?
     
  9. Micnax

    aa Micnax I maek map

    Messages:
    2,056
    Positive Ratings:
    1,382
    It was one of the first custom maps that was made official, so it has a rather large community following behind it. Not to mention it's one of the more played CTF maps just behind 2fort.
     
  10. Geit

    aa Geit 💜 I probably broke it 💜

    Messages:
    597
    Positive Ratings:
    1,112
    Apologies for the fact that I keep bumping this... I've now mostly finished rewriting the Stamps tracker, and it can be found here: http://stamps.geit.co.uk/ -- It will suffer from an extreme drop if Valve ever actually do correct their numbers, but I will reset the database if and when that happens. (or I'll retroactively correct my data, more likely.)

    If anyone would like the source, or access to the raw data, please send me a message
     
    • Thanks Thanks x 4
    Last edited: Jan 9, 2013