Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
1
vote
1answer
35 views

Scikit model is not able to predict sequence correctly

I am trying to create a regression model using scikit-learn for predicting car price. The input data are, car model(trim), kilometers used, past resale price of similar car and age of used car. I am ...
3
votes
1answer
162 views

Difference between learning_curve and validation_curve

What is the difference between these two curves: learning_curve and validation_curve ?
0
votes
2answers
23 views

How to Keep Missing Values in Ordinal Logistic Regression

I’m using mord package in python to do ordinal logit regression (predict response to movie rating 1-5 stars). One of my predictor variables is also ordinal but ...
0
votes
2answers
36 views
1
vote
1answer
57 views

Selecting features for malware analysis

I am trying to build a classifier that detects if I have a malaware by predicting the provenance compiler. To do so I have a dataset composed of assembly code in json format : In particular, I want ...
0
votes
0answers
5 views

SVM, which range to use when normalizing

I am using the SVM classifier from Scikit Learn. I was wondering is there is a know-best-practice when it comes to normalization. I'm using different normalization tecniques, but all my normalized ...
0
votes
0answers
64 views

PCA scikit-learn - ValueError: array must not contain infs or NaNs

I use PCA from from sklearn.decomposition to reduce data dimension. ...
0
votes
2answers
28 views

Sklearn LocalOutlierFactor contamination parameter

Can anyone provide an intuitive explanation of the choice of contamination parameter used in sklearn's LocalOutlierFactor implementation when ...
1
vote
2answers
35 views

Majority voting in scikit-learn Random forest

My main concern is that i need to understand that how does the random forest do majority voting in scikit learn source code. I did not find that specific code in source code of RandomForest. if ...
4
votes
1answer
67 views

TS-SS and Cosine similarity among text documents using TF-IDF in Python

A common way of calculating the cosine similarity between text based documents is to calculate tf-idf and then calculating the linear kernel of the tf-idf matrix. TF-IDF matrix is calculated using ...
2
votes
0answers
14 views

Temporal outlier Analysis on sensor data

I am working to find anomaly/outliers in sensor data using unsupervised machine learning (without training dataset). I have around 20000 samples taken per minute of various sensors. I just need to ...
0
votes
0answers
17 views

Workaround on using Grid Search when we have scipy.sparse.csr.csr_matrix

I am reviewing some techniques based on scikit-learn and I would like to check what are the best parameters for SVM using Grid Search. The thing is that I don't know how to use Grid Search to the ...
1
vote
0answers
18 views

How to use a custom objective function?

The following is what I'm trying to accomplish: I have a charity contact data set. Each contact has features such as sex, age and so on, which we define as X. Now we are doing a solicitation ...
1
vote
1answer
34 views

What to do with large number of collinear variables?

I have this time-series dataset that has 63 features, out of which 57 were manually engineered. While checking for collinearity, I get this correlation matrix: As can be seen there are a number of ...
0
votes
0answers
6 views

Pipeline that cached the results

I use pandas to do feature extraction for machine learning. I hope to achieve the following: Consider I have five data processing steps done sequentially, and I execute it once, the results will be ...
0
votes
1answer
26 views

Explaining feature_importances_ in Scikit Learn RandomForestRegressor

For a project, I used the feature_importances_ attributes from the RandomForestRegressor. Everything works well but I don't know ...
1
vote
1answer
71 views

LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
0
votes
3answers
35 views

Cross-fold validation done on whole dataset or training set?

I have a dataset of 77 samples with 302 features with two labels (0,1). I trained an SVM with gridsearch (cv=5) to perform binary classification. In one run of my script, I do a test-train split, ...
0
votes
0answers
49 views

Value Error: MSLE & CrossVal

I'm trying to run cross validation with mean squared log error with sklearn and getting the following error message: ...
0
votes
1answer
35 views

What is the use of fit method in sklearn.preprocessing.Normalizer()?

According to the documentation of fit(self, X[, y]) method of sklearn.preprocessing.Normalizer(), it does nothing and return the estimator unchanged. I understand that if I intend to normalize data I ...
0
votes
3answers
125 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
0
votes
3answers
117 views

Gradient decent in Python

I just finished working on my first machine learning algorithm i.e Linear regression. I want to reduce the rmse by optimising the model. I found out that gradient decent does the same job. But i dont ...
0
votes
0answers
21 views

Segment Data in SKlearn Pipeline

I have one dataset, and I would like to split this dataset into two segments - representing two different groups of people. I have created two separate binary classifiers for these two groups. What ...
1
vote
2answers
95 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
2
votes
1answer
34 views

Hierarchical Clustering: Extract observations from large heatmap

I'm currently trying to visualize a large data set as heat map. That in itself works smoothly but I struggle with gaining insights from interestingly looking clusters. Specifically, I have two ...
0
votes
0answers
17 views

Feature importance in SVM

Why is there no command for feature importance in SVM like the one provided in Random Forest feature_importance_ from ...
0
votes
1answer
40 views

'PCA' object has no attribute 'explained_variance_'

Elbow Method - Finding the number of components required to preserve maximum variance. My code: ...
0
votes
1answer
83 views

Error encoding categorical features using sklearn pipelines

I am new to sklearn pipelines and am using this post as a guide for my code: https://www.codementor.io/bruce3557/beautiful-machine-learning-pipeline-with-scikit-learn-uiqapbxuj I am trying to encode ...
0
votes
0answers
24 views

IndexError: too many indices for array

Attempting to load a dataset and print out a prediction for each array, yet I keep getting an IndexError.... ...
0
votes
0answers
17 views

How to use countVectorizer and TfidfVectorizer?

I am new to data science and trying to figure things out. I have been asked to create a countVectorizer with binary counts of 1-grams and TfidfVecortizer with 1-gram, both avoiding English stop words. ...
0
votes
0answers
23 views

How can I map the sample from the original feature space to the new kernel feature space? (Sk-learn)

Let's say I have a very basic SVM model, implementin sk-learn: clf = SVC(kernel='rbf', class_weight=weights, gamma=gamma) clf.fit(X,y) X is the sample space with ...
0
votes
0answers
16 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
2
votes
2answers
263 views

Advice and Ideas appreciated - Machine Learning one man project

I have a project where I am supposed to start from scratch and learn how machine Learning works. So far everything is working out better than expected but I feel as I am offered to many ways to choose ...
0
votes
0answers
10 views

Recursive feature elimination on train data or complete dataset and dummy encoding

I am using RFE with logistic regression. I will also be doing cross validation with RFE (RFECV in sklearn) to get the optimum number of features. I am not sure whether to use RFECV on just train ...
0
votes
1answer
57 views

Is it necessary to convert labels in string to integer for scikit_learn and xgboost?

I have a tabular data with labels that are in string. I will feed the data to decision trees in scikit_learn and XGBoost classifier. Is it necessary to convert the labels in string to integers for ...
0
votes
0answers
7 views

How can I predict geographical neighborhoods in a city by coords and income

I have a dataset of households in a city with family income and latitude and longitude. I would like predict virtual zones or neighborhoods with boundaries, grouping close families with similar income....
0
votes
0answers
9 views

How to upload a sklearn SVM model as a chrome extnesion?

I have trained an SVM/Logistic regression machine learning model using its scikit implementation. But now I want to do the same with Tensorflow/Keras. This is for easy conversion to Tensorflow.js. ...
0
votes
1answer
79 views

If I have negative and positive numbers for a feature, should MinMaxScaler be -1 to 1?

I have a variable X with values ranging from -150 to 400. All the other variables in my training set are positive so I normalized them to be from 0 to 1, or they’re already binary, or they had a ...
3
votes
1answer
57 views

sklearn.metrics.average_precision_score getting different answers for same data but different formats

I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet. Consider the following two ...
0
votes
0answers
38 views

Same coefficient in multivariate regression with dummy variables

Hello Data Science community, I have a model with 1 quantitative variable (y) and 2 categorical variables. In order to work with the categorical variables I have created dummy variables (binary) for ...
0
votes
0answers
49 views

How to plot k-means scatter plot on Standardize Data for 16 features in python? Is it even possible?

I have 16 features in my dataset:['age','job', 'marital', 'education', 'default','balance','housing','loan','contact','day','month','duration','campaign','pdays','previous','poutcome']. And result as :...
0
votes
0answers
9 views

Quantile Regression is inconsistent (lower quantiles predicting higher values at times)

While using scikit-learn's GradientBoostingRegressor's "quantile" loss, I noticed that when I try different values of q to fit the data at 0.05 (5%) increments, there are instances when predicting a ...
0
votes
1answer
56 views

Error in SimpleImputer

...
1
vote
1answer
19 views

Functions in scikit that detect outliers automatically?

I know a way to visualize outliers is to make a box plot, but wanted to know if scikit had any quick ways to detect outliers for each variable?
0
votes
2answers
306 views

How can I replace outliers with maximum non-outlier value?

I am doing univariate outlier detection in python. When I detect outliers for a variable, I know that the value should be whatever the highest non-outlier value is (i.e., the max if there were no ...
1
vote
0answers
30 views

How to tune / choose the preference parameter of AffinityPropagation?

I have large dictionary of "pairwise similarity matrixes" that would look like the following: similarity['group1']: ...
0
votes
2answers
188 views

ValueError: Expected 2D array, got 1D array instead:

I was following this example online for simple text classification And when I create the classifier object like this ...
0
votes
2answers
82 views

Choosing best predictors neural networks

there, I'm currently working on a project where my database has about 120 patterns with 39 columns and I am trying to build a predictive neural network out of it. It's regression task. I was trying ...