Regression for discrete values?

Question

I am a novice in machine learning/statistical algorithms, but I have worked with some simple classifiers and regression.

I would like some opinions on whether I am going the right direction or not, given my limited knowledge.

My problem is the following: I would like the best prediction for the price of an input item given the known price of the most similar items in a database.

At first, this sounded like regression problem, but my features are both discrete and contiguous. For example, item make/model is discrete while year of manufacture is contiguous?

Now, I am thinking of finding $x$ closest matches via Euclidean distance, and then weighted average + somehow making sure the certain feature match exactly (like item model type).

Thanks.

Answer 1

You can still perform regression even if your input (or part of it) is discrete. If you think of it, even your "continues" values are actually discrete (starting from their initial measurement accuracy/resolution).

You need to decide if you want to train a model to perform estimation, or just cast your input features to a vector space and use them with some euclidean distance (to make sure some features match exactly, just add some "if" check so you can calculate distances only to relevant samples).

To train a model, it should except as input all the features (preferably normalized), and output a single number which you can later round to the closest discrete value (if you like).

Answer 2

I highly recommend starting simple. It seems like you're trying to jump into some sort of recommendation problem without even knowing your data or what's possible.

---

If you want to predict price, this is a regression problem. There are regression problems and classification problems. Basically, given some features (discrete (car model) or continuous (Miles per Gallon)) you want to estimate the price (a continuous variable).

Your model will use the independent variables (your features) to estimate the dependent variable. For example, a linear regression model is of the form $y=mx + b$ or $y=\beta_0 + \beta_1x$ (same thing). Let's suppose $x$ represents car model, and we have two car models: Model T and Model S, equal to 0 and 1 equivalently.

When you fit a linear regression model of this type, an intercept is learned for each class of $x$. Thus, we might have something like $\beta_0 = b = 10000$. The model also determines the slope of the model, say $\beta_1 = m = 5000$.

Therefore, when we want to predict the price of a car, we use the full linear regression model: $$ y = 10000 + 5000x $$

Consequently, a Model T will be estimated to be 10000 and a Model S will be estimated at 15000.

---

Now, using this knowledge, we can extend to a more general problem. You have some set of features $X$ and a continuous prediction $y$. There are many regression models to use, linear regression (using ordinary least squares) is one. Scikit-learn has various regression models to choose and tune, and you can find more on their site. Most, if not all, follow the basic idea outline above.

I recommend going and trying out linear regression. It'll probably perform poorly, but that's ok! It's a good baseline to expand off. I don't really understand the recommendation system you're thinking of and how that ties in, so I can't say how to handle that. But it seems like you want to recommend a price given some set of features, which is what regression does.

As for preprocessing your data, scikit-learn has lots of tutorials for that and I recommend Googling phrases like "encode categorical features" or "one-hot encoding" or "set up features for linear regression". This is what is known as preprocessing.

Hopefully this wasn't too dumbed down, it was difficult to tell what you are familiar with and if anything will be seen by others and be found useful.