Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
8
votes
3answers
7k views

Nested cross-validation and selecting the best regression model - is this the right SKLearn process?

If I understand correctly, nested-CV can help me evaluate what model and hyperparameter tuning process is best. The inner loop (GridSearchCV) finds the best ...
8
votes
2answers
12k views

How to calculate the fold number (k-fold) in cross validation?

I am confused about how I choose the number of folds (in k-fold CV) when I apply cross validation to check the model. Is it dependent on data size or other parameters?
8
votes
3answers
16k views

What is the difference between CountVectorizer token counts and TfidfTransformer with use_idf set to False?

We can use CountVectorizer to count the number of times a word occurs in a corpus: ...
8
votes
2answers
8k views

How to use Cohen's Kappa as the evaluation metric in GridSearchCV in Scikit Learn?

I have class imbalance in the ratio 1:15 i.e. very low event rate. So to select tuning parameters of GBM in scikit learn I want to use Kappa instead of F1 score. My understanding is Kappa is a better ...
8
votes
2answers
8k views

Why does Gradient Boosting regression predict negative values when there are no negative y-values in my training set?

As I increase the number of trees in scikit learn's GradientBoostingRegressor, I get more negative predictions, even though there are no negative values in my ...
8
votes
4answers
18k views

Clustering for mixed numeric and nominal discrete data

My data includes survey responses that are binary (numeric) and nominal / categorical. All responses are discrete and at individuals level. Data is of shape (n=7219, p=105). Couple things: I am ...
8
votes
2answers
6k views

When to use Standard Scaler and when Normalizer?

I understand what Standard Scalar does and what Normalizer does, per the scikit documentation: Normalizer, Standard Scaler. I know when Standard Scaler is applied. But in which scenario is Normalizer ...
8
votes
1answer
4k views

How max_features parameter works in DecisionTreeClassifier?

What is the parameter max_features in DecisionTreeClassifier responsible for? I thought it defines the number of features the ...
8
votes
1answer
1k views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
8
votes
1answer
974 views

sklearn - overfitting problem

I'm looking for recommendations as to the best way forward for my current machine learning problem The outline of the problem and what I've done is as follows: I have 900+ trials of EEG data, where ...
7
votes
2answers
5k views

Struggling to integrate sklearn and pandas in simple Kaggle task

I'm trying to use the sklearn_pandas module to extend the work I do in pandas and dip a toe into machine learning but I'm struggling with an error I don't really understand how to fix. I was working ...
7
votes
6answers
9k views

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the ...
7
votes
1answer
14k views

cosine_similarity returns matrix instead of single value

I am using below code to compute cosine similarity between the 2 vectors. It returns a matrix instead of a single value 0.8660254. [[ 1. 0.8660254] [...
7
votes
2answers
9k views

Using TF-IDF with other features in SKLearn

What is the best/correct way to combine text analysis with other features? For example, I have a dataset with some text but also other features/categories. SKlearn's TF-IDF vectoriser transforms text ...
7
votes
1answer
6k views

Multiple Categorical values for a single feature how to convert them to binary using python

I have a data set of movies which has 28 columns. One of them is genres. For each row in this data set, the value for column genres is of the form "Action|Animation|Comedy|Family|Fantasy". I want to ...
7
votes
5answers
12k views

DeprecationWarning: The 'categorical_features' keyword is deprecated in version 0.20

I was watching Machine Learning A- Z from SuperDataScience but when I was doing below code sample: ...
7
votes
1answer
1k views

Naive Bayes Should generate prediction given missing features (scikit learn)

Seeing that Naive Bayes uses probability to make a prediction, and treats features as being conditionally independent of each other, then it makes sense that the model can still make a prediction ...
7
votes
1answer
128 views

How to decide how many n_neighbors to consider while implementing LocalOutlierFactor?

I have a data set with rows: 134000 and columns: 200. I am trying to identify the outliers in data set using LocalOutlierFactor from scikit-learn. Although I ...
7
votes
2answers
153 views

migrating to python from R: specific questions

I have been using R and RStudio for prototyping and model building and due to some persisting problems (which would only be applicable to the environment that I am using in) we have decided to use ...
7
votes
1answer
1k views

Extracting individual emails from an email thread

Most of the open source datasets are well formatted i.e each email message is separated well like the enron email dataset. But out in the real world it is highly difficult to separate a top email ...
7
votes
1answer
4k views

How to use TFIDF vectors with multinomial naive bayes?

Say we have used the TFIDF transform to encode documents into continuous-valued features. How would we now use this as input to a Naive Bayes classifier? Bernoulli naive-bayes is out, because our ...
6
votes
5answers
625 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
6
votes
1answer
211 views

What are your thoughts on SKLearn's dismissal of GPUs for machine learning?

SKLearn has this broad claim in its FAQs: Outside of neural networks, GPUs don’t play a large role in machine learning today, and much larger gains in speed can often be achieved by a careful ...
6
votes
2answers
8k views

Is there a R implementation of isolation forest for anomaly detection?

Is there a R implementation of isolation forest for anomaly detection? Similar to the implementation from sklearn.
6
votes
1answer
3k views

what is the difference between “fully developed decision trees” and “shallow decision trees”?

As reading Ensemble methods on scikit-learn docs, it says that bagging methods work best with strong and complex models (e.g., fully developed decision trees), in contrast with boosting methods ...
6
votes
1answer
1k views

Cross validation for highly imbalanced data with undersampling

In my problem, I am dealing with a highly imbalanced data set, say for every positive class there are 10000 negative one. A normal starting method to train a model is to undersample the data. In this ...
6
votes
2answers
6k views

How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
6
votes
1answer
3k views

how to make sklearn pipeline using custom model?

I want to make a sklearn pipeline using the custom Artificial Neural Network I already have. I want to make pipeline in which input goes to ANN and its output goes to the sklearn.svm.SVC model and ...
6
votes
2answers
5k views

Predicting probability from scikit-learn SVC decision_function with decision_function_shape='ovo'

I have a multiclass SVM classifier with labels 'A', 'B', 'C', 'D'. This is the code I'm running: ...
6
votes
2answers
5k views

TF-IDF vectorizer doesn't work better than countvectorizer

I am working on a multilabel text classification problem with 10 labels. The dataset is small, +- 7000 items and +-7500 labels in total. I am using python sci-kit learn and something strange came up ...
6
votes
3answers
2k views

Linear kernel in SVM performing much worse than RBF or Poly

When trying to train a SVM on some Kaggle data, I have encountered a situation where the linear kernel fails to give any results. This doesn't make sense to me because the RBF kernel works just fine, ...
6
votes
3answers
1k views

Classifier and Technique to use for large number of categories

I am designing a scikit learn classifier for a sequence labelling task which has 5000+ categories and training data is at least 80 million and may grow upto an additional 100 million each year. I have ...
6
votes
1answer
3k views

Applying dimensionality reduction on OneHotEncoded array

I have a really large data set with mixed variables. I have converted categorical variables to numerical using OneHotEncoding and it has resulted in more than a ...
6
votes
1answer
787 views

Interpreting the results of randomized PCA in scikit-learn

I'm using scikit-learn to do a genome-wide association study with a feature vector of about 100K SNPs. My goal is to tell the biologists which SNPs are "interesting". RandomizedPCA really improved ...
6
votes
2answers
5k views

Is there a way of performing stratified cross validation using xgboost module in python?

I am training and predicting on the same data-set, but I want to perform 10-fold cross-validation and predict on the left out fold and thus predict on the whole data set. How can I do this? The ...
6
votes
3answers
4k views

Anyway to know all details of trees grown using RandomForestClassifier in scikit-learn?

I am building a standard RandomForest classifier (named model, see the code below) using scikit-learn package. Now, I want to get all parameters of one Randomforest classifier (including its trees (...
6
votes
3answers
127 views

How to use a one-hot encoded nominal feature in a classifier in Scikit Learn?

I'm working on a genre classification problem on a songs dataset. Since genre is a nominal feature, I used sklearn's LabelBinarizer to get the one-hot encoding for this feature for every row in the ...
6
votes
3answers
408 views

What is difference between Multi-class One vs All and Multilabel Classification?

Although Multi class is different from Multi label classification, whats difference does adding One vs All make in Multi-class. Edit 1: http://scikit-learn.org/stable/modules/multiclass.html#...
6
votes
2answers
5k views

How to calculate KL-divergence between matrices

Given there are two matrices of dimensionality 100x2 with absolute values ranging from -50 to +50. Is it possible to determine the kl-divergence by applying the entropy algorithm from scipy.stats to ...
6
votes
2answers
123 views

Best way to scale across different datasets

I have come across a peculiar situation when preprocessing data. Let's say I have a dataset A. I split the dataset into A_train ...
6
votes
1answer
3k views

sklearn: SGDClassifier yields lower accuracy than LogisticRegression

I'm participating in the kaggle Iceberg Classifier Challenge, where the idea is to classify whether an object present in a radar image is an iceberg or a ship. I am currently trying to implement ...
6
votes
1answer
778 views

Custom metrics for unbalanced classes problem in RandomForest or SVM

My dataset has highly unbalanced classes ‒ foreground of 30 classes with tens of samples against background set of >100k samples. Classifying foreground class as background is quite OK, while ...
5
votes
2answers
463 views

Does using unimportant features hurt accuracy?

I'm using the scikit-learn gradient boosting classifier found here. If I run the classifier on the same data without seeding the random number generator, I get different feature importances, and ...
5
votes
1answer
4k views

Does increasing the n_estimators parameter in decision trees always increase accuracy

I'm using some ML algorithms (from sklearn lib) and on most of them there is a parameter n_estimators which is (if I understood well) the number of used trees. Intuitevely I would say that the more ...
5
votes
2answers
1k views

How can I use variable length inputs to train a regression model?

I'm working predicting a value $y \in \mathbb{R}$ from the value of $x_{n+1}$, where $n$ is the number of samples ($x_{i \in [1,n]}$) used for training. Each training sample $x_{i}$ is a time series ...
5
votes
2answers
3k views

Naive Bayes: Divide by Zero error

OK this is my first time in ML and for starter I am implementing Naive Bayes. I have Cricket(sports) data in which I have to check whether the team will win or lost based on Toss Won|Lost and Bat ...
5
votes
1answer
870 views

Is shuffling training data beneficial for machine learning?

I was curious to know if shuffling ML training data is beneficial to better results? Sorry not a lot of wisdom here, but I have been reading a post from pythonprogramming.net for this topic. I ...
5
votes
1answer
3k views

How to train ML model with multiple variables?

I am trying to learn Machine Learning concepts these days. I understand in a traditional ML data, we will have features and labels. I have following toy data in my mind where I have features like '...
5
votes
1answer
553 views

Is it valid to include your validation data in your vocabulary for NLP?

At the moment, I am following best practices and creating a "bag of words" vector with a vocabulary from the training data. My cross validation (and test) datasets are transformed using this model, ...
5
votes
1answer
3k views

How can l get 50 % examples in training set and 50% in test set for each class when splitting data?

l have a dataset of 200 examples with 10 classes. l would like to split the dataset into training set 50% and test set 50%. for each class, l have 20 examples. Hence, l would like to get for each ...