Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
9
votes
3answers
8k views

Nested cross-validation and selecting the best regression model - is this the right SKLearn process?

If I understand correctly, nested-CV can help me evaluate what model and hyperparameter tuning process is best. The inner loop (GridSearchCV) finds the best ...
9
votes
2answers
9k views

How to use Cohen's Kappa as the evaluation metric in GridSearchCV in Scikit Learn?

I have class imbalance in the ratio 1:15 i.e. very low event rate. So to select tuning parameters of GBM in scikit learn I want to use Kappa instead of F1 score. My understanding is Kappa is a better ...
9
votes
1answer
35k views

How to do stepwise regression using sklearn? [duplicate]

I could not find a way to stepwise regression in scikit learn. I have checked all other posts on Stack Exchange on this topic. Answers to all of them suggests using f_regression. But f_regression ...
9
votes
4answers
14k views

Interpreting Decision Tree in context of feature importances

I'm trying to understand how to fully understand the decision process of a decision tree classification model built with sklearn. The 2 main aspect I'm looking at are a graphviz representation of the ...
9
votes
5answers
19k views

DeprecationWarning: The 'categorical_features' keyword is deprecated in version 0.20

I was watching Machine Learning A- Z from SuperDataScience but when I was doing below code sample: ...
9
votes
1answer
1k views

Imbalanced data causing mis-classification on multiclass dataset

I am working on text classification where I have 39 categories/classes and 8.5 million records. (In future data and categories will increase). Structure or format of my data is as follows. ...
9
votes
2answers
999 views

Is there a method that is opposite of dimensionality reduction?

I am new to the field of machine learning, but have done my share of signal processing. Please let me know if this question has been mislabeled. I have two dimensional data which is defined by at ...
9
votes
1answer
2k views

Can training label confidence be used to improve prediction accuracy?

I have training data that is labelled with binary values. I also have collected the confidence of each of these labels i.e. 0.8 confidence would mean that 80% of the human labellers agree on that ...
9
votes
1answer
897 views

Feature selection for Support Vector Machines

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
9
votes
3answers
5k views

Export weights (formula) from Random Forest Regressor in Scikit-Learn

I trained a prediction model with Scikit Learn in Python (Random Forest Regressor) and I want to extract somehow the weights of each feature to create an excel tool for manual prediction. The only ...
9
votes
1answer
1k views

sklearn - overfitting problem

I'm looking for recommendations as to the best way forward for my current machine learning problem The outline of the problem and what I've done is as follows: I have 900+ trials of EEG data, where ...
8
votes
1answer
10k views

What's the difference between Sklearn F1 score 'micro' and 'weighted' for a multi class classification problem?

I have a multi-class classification problem with class imbalance. I search the best metric to evaluate my model. Sklearn has multiple way of calculating F1 score. I would like to understand the ...
8
votes
2answers
10k views

Why does Gradient Boosting regression predict negative values when there are no negative y-values in my training set?

As I increase the number of trees in scikit learn's GradientBoostingRegressor, I get more negative predictions, even though there are no negative values in my ...
8
votes
4answers
12k views

How do we standardize arrays with NaN?

I used StandardScaler() to standardize data so far, but this doesn't work with NaNs. None of the other methods I know of (MinMaxScaler, RobustScaler, MaxAbsScaler) work with NaNs either. Are there ...
8
votes
6answers
14k views

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the ...
8
votes
1answer
20k views

cosine_similarity returns matrix instead of single value

I am using below code to compute cosine similarity between the 2 vectors. It returns a matrix instead of a single value 0.8660254. [[ 1. 0.8660254] [...
8
votes
2answers
12k views

Using TF-IDF with other features in SKLearn

What is the best/correct way to combine text analysis with other features? For example, I have a dataset with some text but also other features/categories. SKlearn's TF-IDF vectoriser transforms text ...
8
votes
1answer
7k views

How max_features parameter works in DecisionTreeClassifier?

What is the parameter max_features in DecisionTreeClassifier responsible for? I thought it defines the number of features the ...
8
votes
1answer
9k views

Multiple Categorical values for a single feature how to convert them to binary using python

I have a data set of movies which has 28 columns. One of them is genres. For each row in this data set, the value for column genres is of the form "Action|Animation|Comedy|Family|Fantasy". I want to ...
8
votes
1answer
4k views

How to use TFIDF vectors with multinomial naive bayes?

Say we have used the TFIDF transform to encode documents into continuous-valued features. How would we now use this as input to a Naive Bayes classifier? Bernoulli naive-bayes is out, because our ...
7
votes
2answers
627 views

Is over fitting okay if test accuracy is high enough? [duplicate]

I am trying to build a binary classifier. I have tried deep neural networks with various different structures and parameters and I was not able to get anything better than ...
7
votes
3answers
3k views

Can we remove features that have zero-correlation with the target/label?

So I draw a pairplot/heatmap from the feature correlations of a dataset and see a set of features that bears Zero-correlations both with: every other feature and also with the target/label ....
7
votes
2answers
5k views

Struggling to integrate sklearn and pandas in simple Kaggle task

I'm trying to use the sklearn_pandas module to extend the work I do in pandas and dip a toe into machine learning but I'm struggling with an error I don't really understand how to fix. I was working ...
7
votes
1answer
3k views

what is the difference between “fully developed decision trees” and “shallow decision trees”?

As reading Ensemble methods on scikit-learn docs, it says that bagging methods work best with strong and complex models (e.g., fully developed decision trees), in contrast with boosting methods ...
7
votes
1answer
2k views

Cross validation for highly imbalanced data with undersampling

In my problem, I am dealing with a highly imbalanced data set, say for every positive class there are 10000 negative one. A normal starting method to train a model is to undersample the data. In this ...
7
votes
1answer
4k views

how to make sklearn pipeline using custom model?

I want to make a sklearn pipeline using the custom Artificial Neural Network I already have. I want to make pipeline in which input goes to ANN and its output goes to the sklearn.svm.SVC model and ...
7
votes
2answers
8k views

How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
7
votes
2answers
5k views

Is there a way of performing stratified cross validation using xgboost module in python?

I am training and predicting on the same data-set, but I want to perform 10-fold cross-validation and predict on the left out fold and thus predict on the whole data set. How can I do this? The ...
7
votes
3answers
2k views

How to estimate the variance of regressors in scikit-learn?

Every classifier in scikit-learn has a method predict_proba(x) that predicts class probabilities for x. How to do the same thing ...
7
votes
2answers
3k views

Applying dimensionality reduction on OneHotEncoded array

I have a really large data set with mixed variables. I have converted categorical variables to numerical using OneHotEncoding and it has resulted in more than a ...
7
votes
3answers
239 views

What needs to be done to make n_jobs work properly on sklearn? in particular on ElasticNetCV?

The constructor of sklearn.linear_model.ElasticNetCV takesn_jobs as an argument. Quoting the documentation here n_jobs: int, ...
7
votes
1answer
2k views

Naive Bayes Should generate prediction given missing features (scikit learn)

Seeing that Naive Bayes uses probability to make a prediction, and treats features as being conditionally independent of each other, then it makes sense that the model can still make a prediction ...
7
votes
1answer
840 views

Interpreting the results of randomized PCA in scikit-learn

I'm using scikit-learn to do a genome-wide association study with a feature vector of about 100K SNPs. My goal is to tell the biologists which SNPs are "interesting". RandomizedPCA really improved ...
7
votes
3answers
155 views

How to use a one-hot encoded nominal feature in a classifier in Scikit Learn?

I'm working on a genre classification problem on a songs dataset. Since genre is a nominal feature, I used sklearn's LabelBinarizer to get the one-hot encoding for this feature for every row in the ...
7
votes
1answer
329 views

How to decide how many n_neighbors to consider while implementing LocalOutlierFactor?

I have a data set with rows: 134000 and columns: 200. I am trying to identify the outliers in data set using LocalOutlierFactor from scikit-learn. Although I ...
7
votes
2answers
166 views

migrating to python from R: specific questions

I have been using R and RStudio for prototyping and model building and due to some persisting problems (which would only be applicable to the environment that I am using in) we have decided to use ...
7
votes
1answer
904 views

Custom metrics for unbalanced classes problem in RandomForest or SVM

My dataset has highly unbalanced classes ‒ foreground of 30 classes with tens of samples against background set of >100k samples. Classifying foreground class as background is quite OK, while ...
7
votes
1answer
2k views

Extracting individual emails from an email thread

Most of the open source datasets are well formatted i.e each email message is separated well like the enron email dataset. But out in the real world it is highly difficult to separate a top email ...
6
votes
5answers
681 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
6
votes
3answers
16k views

Poisson regression options in python

I want to predict count data. In my understanding both standard classification and regression are not well suited for this. A poisson or binomial regression algorithm seems to do the trick. I am ...
6
votes
1answer
256 views

What are your thoughts on SKLearn's dismissal of GPUs for machine learning?

SKLearn has this broad claim in its FAQs: Outside of neural networks, GPUs don’t play a large role in machine learning today, and much larger gains in speed can often be achieved by a careful ...
6
votes
2answers
9k views

Is there a R implementation of isolation forest for anomaly detection?

Is there a R implementation of isolation forest for anomaly detection? Similar to the implementation from sklearn.
6
votes
4answers
20k views

How to use SimpleImputer Class to replace missing values with mean values using Python?

This is my code ...
6
votes
1answer
27k views

Number of features of the model must match the input. Model n_features is `N` and input n_features is `X`.

I am new to data science and trying get some results. I'm applying Decision Tree Classifier. When my train and test datasets' size are not equal I get an error `...
6
votes
2answers
6k views

Predicting probability from scikit-learn SVC decision_function with decision_function_shape='ovo'

I have a multiclass SVM classifier with labels 'A', 'B', 'C', 'D'. This is the code I'm running: ...
6
votes
2answers
4k views

Image clustering by similarity measurement (CW-SSIM)

I'm trying to use scikit-learn and pyssim for clustering a set of images - less than 100. The end goal is to place the images into several buckets (clusters) according to the calculated similarity ...
6
votes
1answer
6k views

How to train ML model with multiple variables?

I am trying to learn Machine Learning concepts these days. I understand in a traditional ML data, we will have features and labels. I have following toy data in my mind where I have features like '...
6
votes
2answers
4k views

Naive Bayes: Divide by Zero error

OK this is my first time in ML and for starter I am implementing Naive Bayes. I have Cricket(sports) data in which I have to check whether the team will win or lost based on Toss Won|Lost and Bat ...
6
votes
1answer
901 views

Is it valid to include your validation data in your vocabulary for NLP?

At the moment, I am following best practices and creating a "bag of words" vector with a vocabulary from the training data. My cross validation (and test) datasets are transformed using this model, ...
6
votes
2answers
5k views

TF-IDF vectorizer doesn't work better than countvectorizer

I am working on a multilabel text classification problem with 10 labels. The dataset is small, +- 7000 items and +-7500 labels in total. I am using python sci-kit learn and something strange came up ...

1
2
3 4 5
32