Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
1
vote
0answers
9 views

Temporal outlier Analysis on sensor data

I am working to find anomaly/outliers in sensor data using unsupervised machine learning (without training dataset). I have around 20000 samples taken per minute of various sensors. I just need to ...
0
votes
0answers
12 views

Workaround on using Grid Search when we have scipy.sparse.csr.csr_matrix

I am reviewing some techniques based on scikit-learn and I would like to check what are the best parameters for SVM using Grid Search. The thing is that I don't know how to use Grid Search to the ...
1
vote
0answers
16 views

How to use a custom objective function?

The following is what I'm trying to accomplish: I have a charity contact data set. Each contact has features such as sex, age and so on, which we define as X. Now we are doing a solicitation ...
1
vote
1answer
20 views

What to do with large number of collinear variables?

I have this time-series dataset that has 63 features, out of which 57 were manually engineered. While checking for collinearity, I get this correlation matrix: As can be seen there are a number of ...
0
votes
0answers
4 views

Pipeline that cached the results

I use pandas to do feature extraction for machine learning. I hope to achieve the following: Consider I have five data processing steps done sequentially, and I execute it once, the results will be ...
0
votes
1answer
21 views

Explaining feature_importances_ in Scikit Learn RandomForestRegressor

For a project, I used the feature_importances_ attributes from the RandomForestRegressor. Everything works well but I don't know ...
1
vote
1answer
30 views

LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
0
votes
2answers
22 views

Cross-fold validation done on whole dataset or training set?

I have a dataset of 77 samples with 302 features with two labels (0,1). I trained an SVM with gridsearch (cv=5) to perform binary classification. In one run of my script, I do a test-train split, ...
0
votes
0answers
4 views

Don't understand error message (basic sklearn command) [migrated]

I'm new to Python and programming in general and I wanted to exercise a littlebit with linear regression in one variable. Im currently following this tutorial in the link https://www.youtube.com/...
0
votes
0answers
20 views

Value Error: MSLE & CrossVal

I'm trying to run cross validation with mean squared log error with sklearn and getting the following error message: ...
0
votes
1answer
19 views

What is the use of fit method in sklearn.preprocessing.Normalizer()?

According to the documentation of fit(self, X[, y]) method of sklearn.preprocessing.Normalizer(), it does nothing and return the estimator unchanged. I understand that if I intend to normalize data I ...
0
votes
1answer
22 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
0
votes
1answer
23 views

Gradient decent in Python

I just finished working on my first machine learning algorithm i.e Linear regression. I want to reduce the rmse by optimising the model. I found out that gradient decent does the same job. But i dont ...
0
votes
0answers
20 views

Segment Data in SKlearn Pipeline

I have one dataset, and I would like to split this dataset into two segments - representing two different groups of people. I have created two separate binary classifiers for these two groups. What ...
1
vote
2answers
32 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
2
votes
1answer
31 views

Hierarchical Clustering: Extract observations from large heatmap

I'm currently trying to visualize a large data set as heat map. That in itself works smoothly but I struggle with gaining insights from interestingly looking clusters. Specifically, I have two ...
0
votes
0answers
13 views

Feature importance in SVM

Why is there no command for feature importance in SVM like the one provided in Random Forest feature_importance_ from ...
0
votes
1answer
14 views

'PCA' object has no attribute 'explained_variance_'

Elbow Method - Finding the number of components required to preserve maximum variance. My code: ...
0
votes
1answer
34 views

Error encoding categorical features using sklearn pipelines

I am new to sklearn pipelines and am using this post as a guide for my code: https://www.codementor.io/bruce3557/beautiful-machine-learning-pipeline-with-scikit-learn-uiqapbxuj I am trying to encode ...
0
votes
0answers
15 views

IndexError: too many indices for array

Attempting to load a dataset and print out a prediction for each array, yet I keep getting an IndexError.... ...
0
votes
0answers
13 views

How to use countVectorizer and TfidfVectorizer?

I am new to data science and trying to figure things out. I have been asked to create a countVectorizer with binary counts of 1-grams and TfidfVecortizer with 1-gram, both avoiding English stop words. ...
0
votes
0answers
21 views

How can I map the sample from the original feature space to the new kernel feature space? (Sk-learn)

Let's say I have a very basic SVM model, implementin sk-learn: clf = SVC(kernel='rbf', class_weight=weights, gamma=gamma) clf.fit(X,y) X is the sample space with ...
0
votes
0answers
16 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
2
votes
2answers
228 views

Advice and Ideas appreciated - Machine Learning one man project

I have a project where I am supposed to start from scratch and learn how machine Learning works. So far everything is working out better than expected but I feel as I am offered to many ways to choose ...
0
votes
0answers
6 views

Recursive feature elimination on train data or complete dataset and dummy encoding

I am using RFE with logistic regression. I will also be doing cross validation with RFE (RFECV in sklearn) to get the optimum number of features. I am not sure whether to use RFECV on just train ...
0
votes
1answer
22 views

Is it necessary to convert labels in string to integer for scikit_learn and xgboost?

I have a tabular data with labels that are in string. I will feed the data to decision trees in scikit_learn and XGBoost classifier. Is it necessary to convert the labels in string to integers for ...
0
votes
0answers
7 views

How can I predict geographical neighborhoods in a city by coords and income

I have a dataset of households in a city with family income and latitude and longitude. I would like predict virtual zones or neighborhoods with boundaries, grouping close families with similar income....
0
votes
0answers
9 views

How to upload a sklearn SVM model as a chrome extnesion?

I have trained an SVM/Logistic regression machine learning model using its scikit implementation. But now I want to do the same with Tensorflow/Keras. This is for easy conversion to Tensorflow.js. ...
0
votes
1answer
24 views

If I have negative and positive numbers for a feature, should MinMaxScaler be -1 to 1?

I have a variable X with values ranging from -150 to 400. All the other variables in my training set are positive so I normalized them to be from 0 to 1, or they’re already binary, or they had a ...
3
votes
1answer
47 views

sklearn.metrics.average_precision_score getting different answers for same data but different formats

I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet. Consider the following two ...
0
votes
0answers
28 views

Same coefficient in multivariate regression with dummy variables

Hello Data Science community, I have a model with 1 quantitative variable (y) and 2 categorical variables. In order to work with the categorical variables I have created dummy variables (binary) for ...
0
votes
0answers
16 views

How to plot k-means scatter plot on Standardize Data for 16 features in python? Is it even possible?

I have 16 features in my dataset:['age','job', 'marital', 'education', 'default','balance','housing','loan','contact','day','month','duration','campaign','pdays','previous','poutcome']. And result as :...
0
votes
0answers
7 views

Quantile Regression is inconsistent (lower quantiles predicting higher values at times)

While using scikit-learn's GradientBoostingRegressor's "quantile" loss, I noticed that when I try different values of q to fit the data at 0.05 (5%) increments, there are instances when predicting a ...
0
votes
1answer
30 views

Error in SimpleImputer

...
0
votes
0answers
11 views

Functions in scikit that detect outliers automatically?

I know a way to visualize outliers is to make a box plot, but wanted to know if scikit had any quick ways to detect outliers for each variable?
0
votes
2answers
47 views

How can I replace outliers with maximum non-outlier value?

I am doing univariate outlier detection in python. When I detect outliers for a variable, I know that the value should be whatever the highest non-outlier value is (i.e., the max if there were no ...
1
vote
0answers
26 views

How to tune / choose the preference parameter of AffinityPropagation?

I have large dictionary of "pairwise similarity matrixes" that would look like the following: similarity['group1']: ...
0
votes
1answer
47 views

ValueError: Expected 2D array, got 1D array instead:

I was following this example online for simple text classification https://towardsdatascience.com/machine-learning-nlp-text-classification-using-scikit-learn-python-and-nltk-c52b92a7c73a And when I ...
0
votes
2answers
52 views

Choosing best predictors neural networks

there, I'm currently working on a project where my database has about 120 patterns with 39 columns and I am trying to build a predictive neural network out of it. It's regression task. I was trying ...
2
votes
1answer
22 views

Confusion for considering accuracy or standard deviation in selecting the best parameters

I have a model with a various parameters to test. The size of the dataset I have is not really large (~500 documents). My issue is that when I test the parameters using 10 CV, some of them produce ...
0
votes
1answer
40 views

hyperparameter tuning with validation set

For what I know, and correct me if I am wrong, the use of cross-validation for hyperparameter tuning is not advisable when I have a huge dataset. So, in this case it is better to split the data in ...
2
votes
2answers
49 views

Is it possible to have a default class in multi class classification?

In the general text classification problem, training a machine learning model to detect if a text belongs to one of N number of classes always yields a value in N. Even if the text that was passed to ...
2
votes
1answer
19 views

(Feature selection) In which cases it is legitimate to remove features manually?

I am dealing with the feature in which only one category takes up about 90%, the instances of more than 30 other categories are sparse. Is it reasonable to remove this feature before building an ...
1
vote
1answer
34 views

(Feature Selection) different results from L2-based and Tree-based

I am doing feature selection using Sklearn: Tree-based feature selection : RandomForestClassifier.feature_importances_ L2-based feature selection: LogisticRegression.coef_ Target variable is binary ...
0
votes
1answer
24 views

Transform test data when using a persistent model

I'm quite new to data science and only slowly following the necessary steps to get valid results using scikit-learn. As far as I understand you fit and transform the training data and only transform ...
3
votes
1answer
43 views

Validation Curve Interpretations for Decision Tree

I'm working on a machine learning class, and we're using supervised learning right now, starting with decision trees. I'm using the UCI Credit Card dataset (whether or not certain people will default ...
1
vote
1answer
23 views

(Scikit-learn) differences between LinearSVC, 'linear' kernel SVC and poly kernel SVC with degree 1

I would like to know the differences between: linearSVC() SVC(kernel='lineaer) ...