I had this basic query on ML and would like to get basic ideas on modelling prediction models using ML and Python.

Say I have a training data of 1000 items as

Item_name, Attrib_1, Attrib_2, Attrib_3,....Attrib_N, Cost

And my aim is to create a model to predict cost for a new item given the attributes.

So where should I start and what are the different ways to prediction and solve this problem ? Also how to evaluate different methods ?

  • $\begingroup$ You might find taking your first steps in machine learning easier and faster if you use a specialised graphical environment specifically for ML, such as WEKA. Most of the popular algorithms present in WEKA have Python equivalents, so once you have explored how different algorithms behave you can implement them with appropriate parameters using the python libraries. There is an accompanying book for WEKA that introduces ML concepts. $\endgroup$ – image_doctor Apr 22 '15 at 9:39

The type of ML problem you are trying to solve is a regression problem. Essentially, given your attributes, predict a continuous variable.

There are multiple ways to solve your problem, but (assuming attributes are numerical) they all boil down to "plot your data points, then draw a line through them". The different methods calculate the placement of this predictor based on various different measures of error; the most basic probably being least squares, which aims to minimize the sum of squared errors.

Metacademy is a great website for learning ML related stuff online, because they provide roadmaps that tell you what you need to learn before learning ______, as well as how to learn those prerequisites.

Here is a link to the metacademy page for linear regression, which is where you're going to want to start. I'd recommend watching the coursera videos from Andrew Ng. If you're just getting started with ML, his course is a wonderful introduction. However, a common complaint for the coursera class is it's too shallow, and is to be considered as an overview of machine learning. If you consider yourself a little more mathematically savvy, I hear the Stanford course (also taught by him) may be a more in depth resource for you.

You should implement your own basic linear regression at first, just to get a deeper understanding of how it works. Afterwords, you can just use a python ML library called scikit learn to perform the regression (link here). I strongly recommend you take this approach for all ML algorithms; strive to understand at least how the algorithms work at a basic level. It is far too easy for someone just starting to learn about ML to see all the implemented algorithms in scikit learn go on to use the algorithms as black boxes without understanding what the algorithm is actually doing.

Lastly, in order to evaluate the different methods, you should split your dataset into two different parts; a training set with which to train your models on, then a testing set with which to test how accurate the predictions are. The idea here being that your data is representative of the "wild" data that you'll see, so by purposely hiding some data, you can effectively simulate how the algorithm will perform in "real world" conditions.

Generally, people go with something along the lines of 70% of your data being used to train, and 30% of it being used to test. Later on, as you get into complicated models with multiple parameters, it will help to split your data into 3 parts: training, used to train your model, validation, used to adjust model parameters, and finally, a test set, used to measure how accurate the model is.

Good luck!

  • $\begingroup$ Just an observation, this can be cast as a classification problem, if the cost is binned or the prices fit into common ranges. :) $\endgroup$ – image_doctor Apr 23 '15 at 7:47
  • $\begingroup$ Thanks Henry, suppose if the attributes of items are location name, company name and components name. In this case how can I model ? $\endgroup$ – Sreejithc321 Apr 23 '15 at 12:50
  • $\begingroup$ Hmm, I can't say for certain that I know exactly what you should do. This falls under the domain of regression with categorical variables wiki link here. $\endgroup$ – Henry Qiu Apr 27 '15 at 6:12
  • $\begingroup$ --continuation of last comment-- in which the bulk of the problem lies in translating categorical variables into numerical ones. An easy way to do this would just assign numerical representations to each unique categorical entry. I'm unsure of how this would affect regression. You should be able to train a neural network (basic introduction video link) to do this without a hitch $\endgroup$ – Henry Qiu Apr 27 '15 at 6:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.