21 votes

What would I prefer - an over-fitted model or a less accurate model?

There are a couple of nuances here. Complexity question very important - ocams razor CV - is this trully the case 84%/83% (test it for train+test with CV) Given this, personal opinion: Second one. ...
Noah Weber's user avatar
  • 5,669
15 votes

What would I prefer - an over-fitted model or a less accurate model?

It depends mostly on the problem context. If predictive performance is all you care about, and you believe the test set to be representative of future unseen data, then the first model is better. (...
Ben Reiniger's user avatar
  • 11.6k
13 votes
Accepted

Supervised learning vs reinforcement learning for a simple self driving rc car

I'd suggest you to try a hybrid approach: First, train your car in supervised fashion by demonstration. Just control it and use your commands as labels. This will let you get all the pros of SL. Then,...
rcpinto's user avatar
  • 368
12 votes
Accepted

What kinds of learning problems are suitable for Support Vector Machines?

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non ...
hoaphumanoid's user avatar
11 votes
Accepted

Neural network with flexible number of inputs?

Yes this is possible by treating the audio as a sequence into a Recurrent Neural Network (RNN). You can train a RNN against a target that is correct at the end of a sequence, or even to predict ...
Neil Slater's user avatar
  • 28.8k
10 votes
Accepted

Is max_depth in scikit the equivalent of pruning in decision trees?

Is this equivalent of pruning a decision tree? Though they have similar goals (i.e. placing some restrictions to the model so that it doesn't grow very complex and overfit), ...
Djib2011's user avatar
  • 7,948
9 votes

What would I prefer - an over-fitted model or a less accurate model?

The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. Maybe not. It's true that 100% training accuracy is usually a strong indicator of overfitting, but it's ...
Ray's user avatar
  • 191
8 votes
Accepted

Ideas for prospect scoring model

I faced almost exactly the same scenario a year and a half ago -- basically what you have is a variation of the one-class classification (OCC) problem, specifically PU-learning (learning from Positive ...
Brandon Loudermilk's user avatar
8 votes
Accepted

What is deconvolution operation used in Fully Convolutional Neural Networks?

Upsampling layer is used to increase the resolution of the image. In segmentation, we first downsample the image to get the features and then upsample the image to generate the segments. For ...
Yash Katariya's user avatar
8 votes

Why neural networks do not perform well on structured data?

... someone pointed out that neural networks do not work very well with the structured data (data in tabular format) as compared to the unstructured data (like representing each pixel in an image). ...
Rob's user avatar
  • 513
8 votes
Accepted

Is there any difference between a weak learner and a weak classifier?

A weak learner can be either a classification or a regression algorithm: Boosting (Schapire and Freund 2012) is a greedy algorithm for fitting adaptive basis-function models of the form in ...
Jonathan's user avatar
  • 5,400
7 votes

How to tell if a problem should use regression or classification model?

A good rule of thumb is to look at the level of measurement of the target/response variable. If the response is measured on a nominal scale, the problem is a classification problem. Values on a ...
Johan Falkenjack's user avatar
7 votes
Accepted

Why is there a trade-off between bias and variance in supervised learning? Why can't we have best of both worlds?

The tradeoff between bias and variance summarizes the "tug of war" game between fitting a model that predicts the underlying training dataset well (low bias) and producing a model that doesn't change ...
aranglol's user avatar
  • 2,176
6 votes

Merging sparse and dense data in machine learning to improve the performance

This seems like a job for Principal Component Analysis. In Scikit is PCA implemented well and it helped me many times. PCA, in a certain way, combines your features. By limiting the number of ...
HonzaB's user avatar
  • 1,669
6 votes
Accepted

How are Hyperplane Heatmaps created and how should they be interpreted?

I think I can answer that, since I implement such a thing in my own library, even if I really don't know how it's implemented in other libraries. Although I am confident that if there are other ways, ...
rapaio's user avatar
  • 4,733
6 votes
Accepted

How do linear learning systems classify datapoints that fall on the hyperplane

Linear, binary classifiers can choose either class (but consistently) when the datapoint which is to classify is on the hyperplane. It just depends on how you programmed it. Also, it doesn't really ...
Martin Thoma's user avatar
  • 18.8k
6 votes
Accepted

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

Imagine that your data is not easily separable. Your classifier isn't able to do a very good job at distinguishing between positive and negative examples, so it usually predicts the majority class for ...
timleathart's user avatar
  • 3,930
6 votes

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

Accuracy is probably not a good metric for your problem. For the original dataset, if the model just makes a dummy prediction that all samples belong to the bigger class, the accuracy will be 83% (...
TQA's user avatar
  • 536
6 votes

time series forecasting - sliding window method

Try this: Make the data stationary (remove trends and seasonality). Implement PACF analysis on the label data (For eg: Load) and find out the optimal lag value. Usually, you need to know how to ...
Aniruddh Goteti's user avatar
6 votes

Confused about the different aspects in Machine Learning

Good question and welcome to Datascience Imagine you have the tree as follows. ...
alpha_989's user avatar
  • 161
6 votes
Accepted

Why models performs better If normalize test data and train data separately?

If apply normalization on training and testing in a separate way, I get really good results 85% (and sometimes more) and the further steps I try to do next work better as well. The problem with ...
Erwan's user avatar
  • 25.2k
6 votes

Why does data science see class imbalance as a problem for supervised learning when statistics does not?

It's generally not related to Data Science but what goes around; typically all sort of bad practice relating to laziness / looking for short term rewards. I wouldn't say DS is pushing for it but ...
Lucas Morin's user avatar
  • 2,093
5 votes

How is training data generated

'Training' data is really just splitting data you have already collected into test or training sets. For example, if you want to build a classifier for handwritten numbers, you collect thousands of ...
dmb's user avatar
  • 326
5 votes

What kinds of learning problems are suitable for Support Vector Machines?

Let's assume that we are in a classification setting. For svm feature engineering is cornerstone: the sets have to be linearly separable. Otherwise the data ...
pincopallino's user avatar
5 votes
Accepted

supervised learning and labels

The main difference between supervised and unsupervised learning is the following: In supervised learning you have a set of labelled data, meaning that you have the values of the inputs and the ...
hoaphumanoid's user avatar
5 votes
Accepted

Which supervised learning algorithms are available for matching?

You can try to frame this problem as a recommender systems situation. Where you have your users (prospective students) and items (alumni) and want to recommend to the users one item. It's not a ...
João Almeida's user avatar
5 votes

Binary Classification

Looks like to me this is a classic imbalance binary classification problem (see comments above). What loss are you using ? It looks like your model is predicting the non-membership class because it’s ...
Alexis's user avatar
  • 178
5 votes
Accepted

which metric is better for boosting methods

Depends. The first thing that has to be clear is that you are running an experiment, which means you need to measure both with the same metric. Which one? Depends on which underlying problem you are ...
Juan Esteban de la Calle's user avatar
5 votes

How to represent audio data in a format that can be used for preprocessing and modelling?

Audio .wav codec file has a 44 byte header which will give you critical data like bit depth ( CD quality audio is 16 bits per sample), sample rate ( CD quality uses 44,100 audio samples per second ), ...
Scott Stensland's user avatar
5 votes
Accepted

Where and how to do large scale supervised machine learning?

First, when working with big data most of the time it's more convenient to work with a random subset rather than the whole thing: usually during the design and testing stages there is no need to work ...
Erwan's user avatar
  • 25.2k

Only top scored, non community-wiki answers of a minimum length are eligible