How to move forward on Regression problem

Question

I'm an undergrad interested in machine learning, and I'm playing around with some data in order to get a better understanding of the field.

DATA

I'm working with the following data:

To give you an idea of what you're looking at (sorry for the unlabeled axes) -- each data point corresponds to a basketball player's points per game (y-values) over a one-week period (x-values).

END GOAL

I want to predict progress throughout the year, so in order to test/train, I start with the first two points, fit a regression model, and then predict the third point. I then add the actual value of the third point to the set, re-train, and predict the fourth point, etc.

ISSUES

As you can see, the data is all over the place, and the predictions are just as messy (sometimes getting up to ~200 points per game, which is totally impossible).

I've tested different degrees of linear regression (quadratic, cubic, etc.) and degree=1 is always the best predictor because of how wonky the data is.

IDEAS

I have thought of the following ways to get more accurate predictions:

• Smoothing the data, maybe using a moving average or some variant
• Set an upper limit for predictions
• Non-linear regression

But outside of smoothing the data, I'm not sure if the rest are even possible in a regression model (upper limit) or applicable to this situation (non-linear).

QUESTION

Are any of the ideas I had above worth pursuing? If not, is there anything I should look into that might help me solve this problem?

Thanks!

Answer 1

You have little features (as in none hehe). Your data reminds me a lot of stock market predictions, where you know nothing about what you're studying except for price. Furthermore, price behaves chaotically, because if it was easily predictable, then everybody would be predicting it, thus removing its predictability.

Therefore, if you are serious about studying that data, you may want to look up literature on stock market prediction.

I suggested in a comment that one approach could be to see this as a classification problem rather than a regression problem. This is in fact what many people do in the stock market. Instead of predicting actual price, people study whether the price will go up or down (and therefore whether they should buy or sell).

Here is an example of what I mean, implemented in sklearn:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.cross_validation import cross_val_score
from sklearn.linear_model import LogisticRegression

# random signal
n = 100
t = np.arange(n)
x = np.random.rand(n)
plt.plot(t, x)
plt.show()

# model ups and downs: convert to 0s (down) and 1s (up)
t = t[1:]
y = ((np.sign(x[1:] - x[:-1])+1)/2).astype(int)
plt.plot(t, y)
plt.show()

# one look-ahead model using the four previous observations
X = np.c_[y[:-4], y[1:-3], y[2:-2], y[3:-1]]
y = y[4:]
scores = cross_val_score(LogisticRegression(), X, y, cv=10, n_jobs=-1)
print('Acc: %.2f (%.2f)' % (np.mean(scores), np.std(scores)))

My results:

Acc: 0.75 (0.17)

One advantage of this approach is that it gives you probabilities of going up or down. Remember, a logistic regression is a probabilistic model $P(y|x)$, and here we are taking $P(y|x)<0.5$ to mean down and $P(y|x)>0.5$ to mean up. But if the probability is close to 0.5, you could have your model say "I don't know". This way you only take action when the model reports down or up with high confidence (close to 0 (down) or 1 (up)). A colleague of mine wrote his thesis on this kind of thing. Are you trying to model sports in order to place bets? If so, this is a pretty simple but effective model of choosing when it is worthy to place bets.

But if the idea is to learn about data mining, then I would try another dataset.

• sklearn comes with a bunch of nice datasets
• kaggle has a lot of 101 training competitions rich in tutorials