Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

0
votes
0answers
2 views

How to split a Bunch type

I have a dataset of images saved in python, in a Bunch type. In this buch, one key is 'train' that assume the values 1 or 0. I need to split them in X_train and X_test in a deterministic way (in ...
1
vote
0answers
18 views

Not sure if over-fitting

I trained the data this way : There are four classes , the data distributed evenly (same amount of labels). Used min_max_scaler Used train_test_split(X,y,test_size=0.3,random_state=42,stratify=y) ...
0
votes
1answer
17 views

LSTM : Bad Data or Low preparation

I have three features {feature1, feature2, feature3} the middle have negative sign for some values and i am trying to predict the middle one using LSTM with window size of 10 but i get very high RMS ...
0
votes
1answer
12 views

Classify members of a dataset, given a known group of members

Problem I have a large CSV file with lots of lines. It has 4 columns and each column contains a label and three integers. I know the first 10,000 lines belong to the same group. Now I need to ...
0
votes
0answers
24 views

Which machine learning/deep learning model can I apply to a mix on textual, categorical and numerical data for a binary classification

I have a project based on tweets wherein I am trying to build a binary classifier, I am aware that I can use a contextual LSTM model which takes the metadata of a tweet as an auxiliary input within ...
0
votes
1answer
26 views

Random Forest, Duplicating Data increases Accuracy. Why?

I duplicated my training data for the random forest classifier (Sklearn) and the accuracy of the prediction increased by about 3%. Why?
1
vote
0answers
12 views

How to check the similarity between two transition matrixes

I have two transition matrixes in which the probabilities for the transition between each from state to each to states are ...
0
votes
0answers
17 views

Should I train entirely dataset or new part after I reload a ML model?

I don't know which one is the better way to do. I use scikit to train ML and save it. I train Model M with train data ...
0
votes
1answer
12 views

RGB Image Segmentation using Clustering

I want to apply some segmentation on a dataset for preprocessing purposes. I have tried the "otsu thresholding" approach in order to segment the image. It's a good method, however, I think a ...
0
votes
0answers
12 views

Adaboost error in tuning with base estimator

I am implementing Adaboost for Stacking purpose and i define some methods to train and test model using kfold cross validation but the issue apprears when i used a base estimator for Adaboost i got ...
0
votes
1answer
14 views

How to get different results running sklearn's MeanShift in a single program? (Python3)

I ran into a quirk with sklearn's MeanShift that I don't know how to get around. MeanShift doesn't predictably give the same results on every run, so I wanted to run it multiple times within one ...
1
vote
1answer
46 views

learning curve Sklearn

I was trying Random Forest Algorithm on Boston dataset to predict the house prices medv with the help of sklearn's RandomForestRegressor. Just to evaluate how good ...
0
votes
1answer
22 views

Tensorflow and Sklearn

Is there a way to feed tensorflow tensors into a sklearn model? I have the following model to set up data compression: ...
1
vote
2answers
86 views

How can I implement tangent distance for k-nearest neighbor in python/scikit-learn?

My ultimate aim is to have a function which I can feed into scikit-learn's NearestNeighbor class as a custom metric parameter. ...
0
votes
1answer
28 views

Obtaining a confidence interval for the prediction of a linear regression

The data I am working with is being used to predict the duration of a trip between two points. There are about 100 different trips in the data and ~90k observations. I am using the standard pattern: ...
2
votes
2answers
35 views

How to transform a folder of images in a csv file

I have a folder with a lot of images that I want to use to bild a classificator using a SVM model in python with sklearn. I've always used csv file as train/test set with sklearn, how can I make it? (...
0
votes
1answer
18 views

Forcing a multi-label multi-class tree-based classifier to make more label predictions per document

I'm been experimenting with tree based classifiers for multi-label document classification. All the trees I've created, however, tend to predict only one or two labels per document. Whereas the ...
0
votes
1answer
37 views

Nearest Neighbors on mixed data types in high dimensions

I would like to be able to use nearest neighbors to attempt to find the most similar samples to a subclass of samples (think treated vs untreated) in a dataset with continuous, categorical, and text ...
3
votes
1answer
41 views

How to prefer no choice instead of bad choice with sklearn decision tree

I'm using sklearn decision trees to classify documents in two possible types "type1" and "type2". I've isolated few features that seem pertinent and tried to combine them manually to evaluate the ...
0
votes
1answer
22 views

Random state in machine learning models

I am confused about random_state parameter in some algorithms like AdaboostClasifier, DecisionTree and so on Here is an example ...
0
votes
0answers
3 views

Train OneVsRest svms separately

I need to perform classfication of hundreds of classes. New classes arrive regularly. I also have some large training set (thousands of samples). ...
0
votes
0answers
18 views

Columntransformer multiple columns with vector inputs

This is perhaps more of a coding question than data science so apologies if this is not the right platform to ask this. My question is related to the sklearn's <...
0
votes
0answers
15 views

How to use tokens with sklearn in LDA?

I have a list of tokenized documents containing both unigrams and bi-grams. I would like to perform sklearn LDA on it and have tried the following code: ...
0
votes
1answer
22 views

Parameters used in GridSearchCV Slow Down in skLearn Tutorial

I'm not sure why grid.fit(X,y) is correct, rather than grid.fit(X_2d, y_2d) In this tutorial on RBF SVM Parameters, we are ...
0
votes
0answers
13 views

Feeding machine learning model with different matrix

Well my question is a general question. I tried to find some relevant information before posting my question here, but no success!. I am working on ...
0
votes
1answer
21 views

Handle 50,000 classes in OneVsRestClassifier

I'm new to data science and NLP. I'm trying to solve a problem that is having 1 million rows and some 50,000 distinct classes. The dataset has some text column as a predictor and the other one is the ...
0
votes
1answer
5 views

Spectral clustering result interpretation

I cluster a toy data set into three groups with spectral clustering. Please see the code below. affinity='rbf' by default. ...
0
votes
1answer
51 views

Single machine learning algorithm for multiple classes of data : One hot encoder

I have data of the following kind: x1 x2 y 0 0 1 1 1 0 2 2 2 0 3 3 3 0 4 4 4 1 1 4 5 1 2 8 6 1 3 12 7 1 4 16 Is it possible ...
0
votes
1answer
18 views

Prepare JSON data from sentiment analysis to perform Logistic Regression

I'm new to this field, so very sorry for this basic question. I'm working on a text analysis project using Google's NLP API along with some other APIS. After performing the sentiment analysis I have ...
0
votes
1answer
27 views

Difference between sklearn’s “log_loss” and “LogisticRegression”?

I am a newbie currently learning data science from scratch and I have a rather stupid question to ask. I’m currently learning about binary classification, and I understand that the logistic function ...
0
votes
0answers
10 views

Coefficient of Determination in a 2d space

I am currently working on a project about visual engagement prediction and I would love your help. I need to calculate the Coefficient of Determination $R^{2}$ in order to evaluate if the data that I ...
0
votes
0answers
13 views
3
votes
1answer
38 views

How max_features parameter works in DecisionTreeClassifier?

What is the parameter max_features in DecisionTreeClassifier responsible for? I thought it defines the number of features the ...
1
vote
2answers
28 views

using sklearn class weight to increase number of positive guesses in extremely unbalanced data set?

Hi I have a poorly correlated and unbalanced data set I have to work with. The set is 2 classes, 0 has 96,000 values and 1 has about 200. When I run random forest or other methods I get an output like:...
2
votes
0answers
21 views

Non-linear Support Vector Regression issue - Sklearn Python 3.6

I am fairly new to Sklearn and machine learning and have encountered an issue when using SVR with an RBF kernel. Below is my predicted data compared directly with my real data: I do not know what I ...
1
vote
1answer
15 views

Boostrap parameter in random forest regressor?

There's one parameter in RandomForestRegressor which is bootstrap. By default bootstrap=True bootstrap : boolean, optional (...
0
votes
1answer
11 views

In handwritten digit recognition problem using logistic regression, what changes needed to add another class “Not a Digit”

In handwritten digit recognition problem using logistic regression, normal implementation would forcibly classify even a picture of dog or cat as a digit. To eliminate this, what changes are needed to ...
1
vote
1answer
26 views

Error trying to do a GridSearchCV()

On the following lines of code I am getting ...
0
votes
1answer
11 views

Guidance needed with dimension reduction for clustering - some numerical, lots of categorical data

I've my data in a Pandas df with 25.000 rows and 1.500 columns without any NaNs. Of the columns about 30 contain numerical data which I standardized with StandardScaler(). The rest are cols with ...
2
votes
0answers
12 views

Varying strength of prior for MCMC hierarchical linear model

I am training an MCMC model in using Pymc3. My aim is to build a series of linear regression models which will predict the time to unload a truck, based on the number of crates to unload. I have ...
3
votes
1answer
46 views

Why would a fake feature with random numbers get selected in feature importance?

I'm using a sklearn.ensemble.RandomForestClassifier(n_estimators=100) to work on this challenge: https://kaggle.com/c/two-sigma-financial-news I've plotted my ...
0
votes
0answers
10 views

Opencv kmeans predict equivalent

I'm moving my project from Python's libraries to opencv, and I have one big problem. In Python skylearn, I have kmeans object, which has two useful methods: fit_predict and fit, which both are ...
0
votes
1answer
189 views

DeprecationWarning: The 'categorical_features' keyword is deprecated in version 0.20

I was watching Machine Learning A- Z from SuperDataScience but when I was doing below code sample: ...
1
vote
1answer
16 views

For text classification that has innumerable features, how do I choose the number of neurons and layers for MLPClassifier?

In my use case of text classification (identify the author from a subset of 10 authors), I find that post all processing with trigrams, there are a 100 thousand and odd features with nearly 50k ...
1
vote
0answers
15 views

How to model machine learning problem for cache replacement policy?

I am trying to implement machine learning on Cache Replacement Policy. I want to train a ML model on labelled data acquire from Belady's Optimal Algorithm for Cache replacement policy. For example, ...
0
votes
1answer
21 views

Why is sklearn.metrics.roc_auc_score() seemingly able to accept scores on any scale?

I had input some prediction scores from a learner into the roc_auc_score() function in sklearn. I wasn't sure if I had applied a sigmoid to turn the predictions into probabilities, so I looked at the ...
0
votes
1answer
29 views

Is there any way how to make samples balanced?

I have a dataset which consists of attributes on breakdown of machines.The target variable is machine status which are populated with ones and zeros. The distribution of ones and zeros are given below ...
0
votes
1answer
54 views

What's the difference between Sklearn F1 score 'micro' and 'weighted' for a multi class classification problem?

I have a multi-class classification problem with class imbalance. I search the best metric to evaluate my model. Sklearn has multiple way of calculating F1 score. I would like to understand the ...
0
votes
0answers
36 views

Why precision increases with recall?

I'm working on multi-class classification problem. There are ~1600 samples; 47 features and 9 imbalanced classes. The smallest class includes just 17 samples whereas the larges one over 600 (classes ...
1
vote
1answer
10 views

Nested cross-validation generalization error for multiple models

I am referring to this question: Nested cross-validation and selecting the best regression model - is this the right SKLearn process? In the answers it shows that nested cv can estimate the ...