2
$\begingroup$

I have a function that returns the predicted accuracy of a time-series model. I have two equally-sized numpy arrays, one for the actual direction and one for the predicted direction. I'm classifying whether there is a change in the data's direction from the previous t-1 step. '1' stands for an increase from t-1, '0' stands for no change in direction, and '-1' stands for a decrease from t-1. I'm trying to compare the elements of both arrays to determine if they both contain the same data to determine accuracy.

I can match indexes and count the number of '1s' and '-1s' that match, but I cannot count the number of matching zeros. (It's kinda hard summing zeros). :-) Anyway, I've tried the numpy sum function specifying '0' as the argument for both arrays but it only returns an array of zeros but no count. I'm not trying to create a confusion matrix...the goal here is to create an accuracy score. I plan to take all the matching ones, negative ones and zeros and divide that by the total length to get an accuracy score.

Thanks in advance.

$\endgroup$
  • 3
    $\begingroup$ numpy.array([0,-1, 1]) == np.array([0,-1,1]) will return a boolean array, that you can sum over, is this what you are looking for? $\endgroup$ – Akavall Jul 5 at 23:23
5
$\begingroup$

You can compare the predictions with the expected results directly, using simple comparisons, in this case just ==. This returns boolean values - True or False, which you can sum up because True == 1 and False = 0.

Here is an example for your case using some randomly generated dummy data:

In [1]: import numpy as np                                                                                                           

In [2]: y = np.random.choice([-1, 0, 1], 10)                                                                                         

In [3]: preds = np.random.choice([-1, 0, 1], 10)                                                                                     

In [4]: y                                                                                                                            
Out[4]: array([ 1,  1,  1, -1,  1, -1, -1,  1,  1,  0])

In [5]: preds                                                                                                                        
Out[5]: array([ 0, -1,  1,  0,  1,  1, -1,  1, -1,  0])

The real part that checks where your predictions are correct is then done using two checks:

  1. where are the predictions equal to the ground truth i.e. where were you correct in predicting the direction, and
  2. where are the predictions equal to the direction you are interested in i.e. downwards (-1), no change (0) or upwards (1)

This can be done as follows:

In [6]: (y == preds) & (preds == 0)   #   &   means we needs both checks to be True 
Out[6]:
array([False, False, False, False, False, False, False, False, False, True])

We can see only the final position is True after both these checks, because that is the only place that the prediction was True and the value direction was 0.

You can then write a loop to check all values and do something with them:

In [7]: n = len(preds)    # the number of test samples (= 10 in my dummy example)

In [8]: for direction in [-1, 0, 1]:
            score = sum((y == preds) & (preds == 0)) 
            accuracy = score / n 
            print(f"Direction {direction:>2}: {score}/{n} = {accuracy * 100:.1f}%")

Which gives:

Direction -1: 1/10 = 10.0%
Direction  0: 1/10 = 10.0%
Direction  1: 1/10 = 10.0%
| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your help. $\endgroup$ – brohjoe Jul 6 at 2:50
  • 1
    $\begingroup$ You're welcome, @brohjoe :) If I answered your question, please consider marking it as resolved. $\endgroup$ – n1k31t4 Jul 6 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.