Heatmaps

Geit

💜 I probably broke it 💜
aa
May 28, 2009
598
1,161
Hi all,

This a thread for my attempt at a heat mapping application for map developers. If you've been in chat over the past few weeks, you'll have already seen this in action a few times, this thread is for collecting feedback and ideas for further development.

What's a heat map?

A heat map is a visualization for allowing a designer to infer patterns from various kinds of spatial data according to the relative density of the data. A higher density results in a "hotter" map.

So, how's it progressing?
Currently, I have a working demo available at http://heatmaps.tf if you'd like to have a quick play. We're currently collecting data from all TF2Maps servers and another server host that's allowed us to collect data from them. This application is very much WIP and is in a development environment, so it may not be functioning at all at points.

Great, but I want the raw data!
Raw data is available using the API documented at http://heatmaps.tf/api.html

---


Notice
This is the new thread for Geit's attempt at Heatmaps: the old thread, for InstantMuffin's attempt, is here: http://forums.tf2maps.net/showthread.php?t=22563
 
Last edited:

Geit

💜 I probably broke it 💜
aa
May 28, 2009
598
1,161
So I've been working on this a fair bit over the last few days, I've sorted out a very early WIP page here: http://geit.co.uk:62923 (may not always be up, or working correctly, and it doesn't work on smaller screens).

I need to rewrite parts of what I have (they were written on very little sleep, and are very hacky) then work on the filters implementation. I'm sort of away for the weekend, so may not work on it till next week some time, but it shouldn't take too long after that.
 
Last edited:

InstantMuffin

L2: Junior Member
May 26, 2014
64
48
[...]Since Muffin decided to disassociate himself from us,[...]

That's one way to put it. Precisely I disassociated myself from you and frozen, not from the community. I created a group which people could join to see how much interest there actually is, and with 6 people in it and different stuff I'm working on, it is not my priority. If people want it, they can still have it from me.
 

Bakscratch

Finisher of Maps
aa
Oct 29, 2010
714
1,492
thanks but we now have a sun that does not give light to the moon assuming the moon is going to owe it one

But in the end It doesn't even matter I had to fall to lose it all but in the end It doesn't even matter
 

TyeZenneth

L6: Sharp Member
May 31, 2014
340
293
That's one way to put it. Precisely I disassociated myself from you and frozen, not from the community. I created a group which people could join to see how much interest there actually is, and with 6 people in it and different stuff I'm working on, it is not my priority. If people want it, they can still have it from me.

Except I haven't even heard about this group until now, as I'm sure many people haven't.
 

Crash

func_nerd
aa
Mar 1, 2010
3,315
5,499
So I've been working on this a fair bit over the last few days, I've sorted out a very early WIP page here: http://geit.co.uk:62923 (may not always be up, or working correctly, and it doesn't work on smaller screens).

I need to rewrite parts of what I have (they were written on very little sleep, and are very hacky) then work on the filters implementation. I'm sort of away for the weekend, so may not work on it till next week some time, but it shouldn't take too long after that.

Oh man that looks amazing. Is it all maps tested right now showing up on there or does it have to be "turned on"?
 

Another Bad Pun

In the shadows, he saw four eyes lit by fire
aa
Jan 15, 2011
801
1,845
Jeez, this is really neat. So you've been stalking play-tests this whole time?
Layl pointed out in chat that the engi party on koth_amazing is visible in the heatmaps:

Edit: I really really wish you could zoom into the map somehow
 
Last edited:

Teddy Schnapps

L2: Junior Member
Feb 5, 2014
51
22
It would be very nice to have the separation of self-suicide deaths and regular gameplay deaths. I think that would be more helpful then just any death willy-nilly.

Also I would personally like to see in real time the players moving from spawn throughout the map. This will give me a better understanding of how players react to their environment and how confused they are with the layout.

P.S. Hope this thing comes out soon! I really, really need it, bad!

Love
Teddy
 

Geit

💜 I probably broke it 💜
aa
May 28, 2009
598
1,161
Oh man that looks amazing. Is it all maps tested right now showing up on there or does it have to be "turned on"?

Data is collected automatically - I have a daemon that collects the overview images that I currently have to run manually before the data will appear - the /data/ interface is open at all times if you want to download your kill data.

Jeez, this is really neat. So you've been stalking play-tests this whole time?
Layl pointed out in chat that the engi party on koth_amazing is visible in the heatmaps:

Edit: I really really wish you could zoom into the map somehow

Map zooming is planned - the code for it is already written down in my notes, it's a low-priority at the moment though.

It would be very nice to have the separation of self-suicide deaths and regular gameplay deaths. I think that would be more helpful then just any death willy-nilly.

Also I would personally like to see in real time the players moving from spawn throughout the map. This will give me a better understanding of how players react to their environment and how confused they are with the layout.

P.S. Hope this thing comes out soon! I really, really need it, bad!

Love
Teddy

Self Suicide is completely ignored by the data logger (they give no value, imo). There are plans to record player movements, my main gripe at the moment is recording that data without destroying the server's FPS and bandwidth. I'd need to store the player's individual movement points and then run a line simplification algorithm... tl;dr It's planned but will not be a first-release feature.
 
Last edited:

wareya

L420: High Member
Jun 17, 2012
493
191
Save their position every second or half second, collect it into deltas, save to disk on a thread when they die or periodically? Can you even do threading?
 

InstantMuffin

L2: Junior Member
May 26, 2014
64
48
[...]I'd need to store the player's individual movement points and then run a line simplification algorithm... tl;dr It's planned but will not be a first-release feature.

You're actually over-thinking this.
I'll see what I can do once I'm available again.
 

Geit

💜 I probably broke it 💜
aa
May 28, 2009
598
1,161
You're actually over-thinking this.
I'll see what I can do once I'm available again.

If you wanted a suitable resolution in the data you'd need to collect positions, as wareya said, every few seconds at most: let's say every 2 seconds. Now let's say the server is full and there are 24 people online. A coordinate trio takes 12 bytes of network/storage at least, so that's 144 bytes per second before network overhead.

Assuming a 30 minute map, that's 0.25 MB per map, per playthrough - In terms of data storage that's nothing: easy peasy to store and process - but now let's say a map accumulates 24 hours of playtime with 24 players: that's 12MB of data that now has to be delivered to the client before they can begin plotting the data, or in real terms: around 16 seconds before the client can begin rendering the heatmap. (at 6mbit and assuming no other delays)

Now if you actually processed your data with a line simplification algorithm such as https://hydra.hull.ac.uk/resources/hull:8338 - pre or post storage - you can reduce the data size by up to 80% while still maintaining virtually the same resolution, which means faster rendering, less storage space and benefits all around.

Personally, I'd rather overthink it. Wasting some server cycles is a pittance compared to getting the heatmaps delivered fast.

Also, I'm hardly one to talk but: don't make promises you can't fulfill.
 

InstantMuffin

L2: Junior Member
May 26, 2014
64
48
EDIT:
nvm, you're right.

[...]Also, I'm hardly one to talk but: don't make promises you can't fulfill.

I'm not making any promisses here. I have no obligations. I came into this by looking for something worthwhile and interesting to do and asking Frozen how I could help mappers. And he brought up heatmaps, so I went for it.
Now while things happened between us two and Frozen another project conveniently came up which is high priority for me now as I can reach/help more people with it and I like to work with the people involved. If I'm available again and I think there's a better way of helping mappers analyze their data (compared to what you do) I wouldn't decline doing them a favor and provide them with an alternative.
If you personally don't want my help (I wouldnt be surprised after what and how you said it back then), feel free to ignore it. Remember, I am not doing this for you, frozen, or the heads of tf2maps. I like to help people who bust their asses trying to accomplish something I recognize as art. If I can help them with a bit of science, why not go for it?
 
Last edited:

wareya

L420: High Member
Jun 17, 2012
493
191
TF2 has a speed limit (per axis?), so if anything suddenly goes significantly faster than that for a single frame (teleports) you can just make a new line.

The problem is really that there is just SO MUCH FUCKING DATA. You would need to filter it dramatically for the client: Only paths before mid, only paths carrying intel, only paths up to 10 seconds out of <given spawn>, etc.
 

henke37

aa
Sep 23, 2011
2,075
515
The trick then would be to load data on demand. Tag each path to allow quick filtering and then let the server do the filtering when sending the data. Index the data for extra fast retrieval. Partion the data in separate files for extra quick filtering on very common criteria like time.
 

Geit

&#128156; I probably broke it &#128156;
aa
May 28, 2009
598
1,161
So while I haven't got much work done over the past week because I've been at a friend's house, I did manage to pick up http://heatmaps.tf/ and move the development site to it. Yey :3