this is my first question in this Exchange so let me know if I should reformat anything.
I am working with 2D images of dimensions (w,h). I collect data from users and sample their activity on the image in question. Each user's activity is sampled and stored in an array data (n,3) where 'n' is the number of points sampled (varying for every user). Each sample is of the format (xi,yi,1) where (xi,yi) represents the coordinates in the image the user currently acts on and '1' is a constant value describing the intensity of his activity. Obviously, since a user can act more than once on the same (xi,yi) pair, the data array does not contain unique values. I then use some custom metrics to estimate the accuracy % of the data collected.
To sum up so far, multiple users act on an image and their activity is recorded in a separate array data_u (for user). Each data_u has a corresponding accuracy % based on custom metrics.
My first goal is to create a heatmap FOR EACH USER'S ACTIVITY. I have read a lot of implementations regarding this issue and decided to create an array h_i of dimensions (w,h). This array is a mask for the original image. Each element in this mask is in the range [0,1] describing, once again, the intensity of the user's activity. In order to account for possible errors, accuracy is used to create a circular area (radius r which is calculated via the accuracy and the height 'h' of the image) of decreasing intensity (think of a Gaussian distribution in a range [0,r] -> [1,0]) for each point in the data_u array. The results are really accurate.
I then go on and apply a similar approach and create a concatenated heatmap. This heatmap uses information form all the users and depicts the average behavior of the users so far.
The problem occurs when I try to create a metric to measure in what percentage can a new random user's activity be predicted based on the data collected so far. A thorough research on this, suggests that I use R-squared (coefficient of determination) as a metric but I am unsure as to how to implement this. I can either use, data_u and accuracy plus a new user or the concatenated heatmap/mask plus a new user as input for the R-squared calculation but at this point I'm lost.
Any ideas as to how to calculate this metric or any suggestions on other metrics I could research?
Thanks in advance, feel free to ask for more details, I will edit this post as needed.