Questions tagged [scikit-learn]

scikit-learn is a popular machine learning package for Python that has simple and efficient tools for predictive data analysis. Topics include classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.

Filter by
Sorted by
Tagged with
1
vote
2answers
18 views

Deep Neural Network Model in sklearn Pipeline

Is it possible to add a deep neural network model as the estimator/model in an sklearn Pipeline? or is it only possible for ML models as the estimator. For example, ...
1
vote
0answers
11 views

How should you preprocess the data before K-fold validation?

I often see Kaggle notebook authors preprocessing the entire training data prior to splitting it for K-fold validation, but does this have a risk of leaking information into the validation set each ...
1
vote
2answers
157 views

Metrics values are equal while training and testing a model

I'm working on a neural network model with python using Keras with TensorFlow backend. Dataset contains two sequences with a result which can be 1 or 0 and positives to negatives ratio in dataset is 1 ...
1
vote
0answers
23 views

How to implement kfold and cv into Hybrid feature selection and evaluate the classification model performance?

I have been working on a Hybrid feature selection combined with hyperopt package for hyperparameter tuning and I am thinking about evaluating the performance of several model classifiers. I looked ...
2
votes
4answers
85 views

Are there ML Libs in Python robust to missing data?

So I was searching on how to handle missing data and came across this post from Machine Learning Mastery. This article states that some algorithms can be made robust to missing data, such as Naive ...
4
votes
3answers
517 views

Best platform to work with when having millions of rows in dataframe

I have table with around 20 features and millions of observations (rows). I need to create model base on this table, however, as it is huge, training models like random forest or XGB takes forever. I'...
1
vote
0answers
15 views

Is there a "tree-based-correlation" for tree-based algorithms?

Although correlated features are not a big issue when training tree-based models, they spoil model explainability. When several features correlate, sometimes they may be picked at random. Then their ...
4
votes
2answers
619 views

Can I do incremental learning with the sklearn implementation of Linear Discriminant Analysis

I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn ...
1
vote
2answers
1k views

Feature scaling for MLP neural network sklearn

I am working with a dataset where the features have multiple scales. Before running scikit-learns's MLP neural network I was reading around and found a variety of different opinions for feature ...
0
votes
1answer
91 views

Creating quality data with sklearn.datasets.make_classification

I'm doing some experiments on some svm kernel methods. My methodology for comparing those is having some multi-class and binary classification problems, and also, in each group, having some examples ...
1
vote
1answer
3k views

hyperparameter tuning with validation set

For what I know, and correct me if I am wrong, the use of cross-validation for hyperparameter tuning is not advisable when I have a huge dataset. So, in this case it is better to split the data in ...
2
votes
1answer
39 views

Reading the classification report of evaluation metric?

I am using the "classification_report" from: from sklearn.metrics import classification_report in order to evaluate a classification model. How can I ...
1
vote
1answer
36 views
0
votes
1answer
34 views

When should I use 'rbf' and 'polynomial' kernel trick in machine learning algo?

I have a problem about hate-speech classification using support-vector machine algorithm. The task is to identify the sentence that contains 'positive' or 'negative' sentiment. Which is the best ...
0
votes
2answers
47 views

How do I fix mis-rendered matplotlib?

How do I correct my data or format it so that it is presentable, and fix my graphs? Dataset is 345551 rows × 7 columns. I am using numpy, pandas, seaborn and matplot lib. It seems that my pricing ...
0
votes
2answers
643 views

Random Forest - Explanation Parameter

I got some question about the "standard" parameter from a random forest. Following I write my understanding about these parameters. I would be glad if I could confirm my understanding or correct it. :)...
0
votes
1answer
36 views

How to identify/recognize that a sentence about talks about future?

Brief Introduction: I have a report/paragraph in which there are sentences with reference to future plans/outlooks/expectations for a particular entity. I want to extract all such sentences for now. ...
1
vote
2answers
3k views

How to run AgglomerativeClustering on a big data in python?

I run AgglomerativeClustering on a sample of data and fit a model. then I decide to predict this fit for all of my data but I got MemoryError. How can I run ...
2
votes
2answers
173 views

How to decode encoded labels in Decision tree classifier

I have some dataset with procurements of organization where actually i'm working. The aim is to find most important features that describe why some processes of purchases is succesful, and why not ...
2
votes
1answer
25 views

How can I get the dataframe after scikit pipeline?

I'm making many data transformations and fitting a model using scikit pipeline, but I need to extract X_train and X_test right after transformations (imputer, encoding, etc) in order to use it for ...
0
votes
0answers
20 views

Different AUC values for xgb and sklearn built in functions [duplicate]

The model is trained with early stopping on a validation set: ...
1
vote
2answers
35 views

Understanding Sklearns learning_curve

I have been using sklearns learning_curve , and there are a few questions I have that are not answered by the documentation(see also here and here), as well as questions that are raised by the ...
1
vote
1answer
185 views

How to structure my data into features and targets for PCA on Big Data?

I want to apply the PCA algorithm from Scikit-Learn.(https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html ) At the part where I have to separate the features and the ...
2
votes
1answer
2k views

Variance threshold with python problem

i’am a beginner in scikit-learn and i’ve a little problem when using feature selection module VarianceThreshold, the problem is when i set the variance ...
0
votes
1answer
32 views

Unsupervised Learning::Satellite Images::Single Bands

Has anyone has success with building models using KMeans for classification? I have images that only have one band and it continues to fail. My guess is that the issue is with both size of the image ...
0
votes
1answer
23 views

Why scikit-learn's sequential feature selection requires how much features to be selected beforehand?

From the version 0.24, the scikit-learn has new method 'SequentialFeatureSelector', which adds (forward selection) or removes (backward selection) features to form a feature subset in a greedy fashion....
0
votes
0answers
14 views

Why doesn't SVC in Sklearn have n_jobs hyperparameter?

Why doesn't SVC in Sklearn have n_jobs hyperparameter unlike other algorithms such as Randomforest or Logistic Regression?
2
votes
2answers
333 views

Unable to understand the usage of labels argument in sklearn.metrics.f1_score

I am trying to model a dataset with RandomForest Classifier. My dataset has 3 classes viz. A, B, C. 'A' is the negative class ...
3
votes
2answers
59 views

How to use sklean pipeline to deal with data that read in line by line

The problem I'm facing is that my data is too big, i can't load it to a dataframe and then process it. However, I really want to use the sklearn pipeline API, so that I can reuse those subclass ...
4
votes
1answer
734 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
0
votes
1answer
164 views

Plotting Polynomial Regression?

I'm reading through Hands-On Machine Learning with Scikit-learn and Tensorflow by Geron. I am creating a simple polynomial regression using sklearn's ...
0
votes
1answer
35 views

Predicting the test data with LinearRegression model gives ValueError: shapes (8523,1606) and (1605,) not aligned: 1606 (dim 1) != 1605 (dim 0)

Fitting the model, testing and getting the score or r2 does not give the error. But when I try to predict the actual data I get this ValueError: ...
0
votes
1answer
560 views

splitting into train test by train_test_split of float values?

How to split into train test by train_test_split of float values ? I used LabelEncoder but I have about 300K lines and when I used the cross_val I saw ...
0
votes
1answer
5k views

bad input shape (5634, 2)

I tried everything and I am not sure how to resolve the following error: ValueError: bad input shape (5634, 2) This is my first machine learning example so please bear with me. This is the python ...
0
votes
2answers
346 views

How to do predict a new sms to be spam or not?

I have trained a model for spam classification - This is my code - ...
0
votes
1answer
46 views

How to use Cross Validation data set?

I am new to data science, and the dataset I am working on is divided into train set, test set, and validation set. However, till now I was splitting the data with ...
2
votes
1answer
254 views

Question about the Silhouette number for K means clustering

I have used K means clustering. In order to find the best value for K, I've looked at the changes of inertia value vs K and also changes of average Silhouette number vs K. The graph for inertia seems ...
2
votes
1answer
91 views

What are some good models to test the speed of a data science machine?

I'm writing a battery of tests (in Python) for the purpose of measuring the speed of my company's different computational instances. The goal is to see how fast different AWS EC2 instances are at ...
2
votes
3answers
2k views

why to use Scaler.fit only on x_train and not on x_test for normalizing value using MinMaxScaler?

while normalising the data everone is saying that we need to fit only on x_train and not on x_test ? why is that we should not fit x_test ? if we should not fit the scaler on x_test then why we need ...
0
votes
2answers
170 views

AttributeError: 'numpy.ndarray' object has no attribute 'fit' [closed]

I am relatively new to ML and in the process of learning pipelines. I am creating a pipeline of custom transformers and get this error: AttributeError: 'numpy.ndarray' object has no attribute 'fit'. ...
1
vote
1answer
2k views

What is difference between leave one subject out and leave one out cross validation

What is the difference between leave one subject out cv and leave one out cross validation (loocv)? are they same or different?. I have images of 24 subject and according to literature, leave one ...
0
votes
0answers
10 views

Increase dimension of dataset for modelisation

I have a dataset like that : | A header | Another header | | -------- | -------------- | | a | {player_1: stat_1,stat_2,stat_3} | | b | {player_2: stat_1,stat_2,stat_3} | | ...
0
votes
1answer
548 views

SKLEARN Metrics report "Number of classes, 28, does not match size of target_names, 35. Try specifying the labels parameter"

What's the proper way to define the labels or target names of classification_report? I have the report like this: ...
1
vote
1answer
4k views

Same TF-IDF Vectorizer for 2 data inputs

I am trying to work on Dataset released by quora, to identify if Question1 has similar intent as of Question2 The dataset looks like: id|question1|question2|is_duplicate 0|What is the step by step ...
6
votes
1answer
113 views

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results ...
2
votes
1answer
35 views

How to fit a KNN and then a linear regression with those neighbors?

How do I fit a KNN to get the $k$ nearest neighbors and then aggregate the those neighbors into a fit using a linear regression (instead of a weighted average) in Scikit-Learn? I've tried creating a ...
0
votes
1answer
27 views
2
votes
3answers
829 views

Scikitlearn grid search random forest using oob as metric?

Have looked at data on oob but would like to use it as a metric in a grid search on a Random Forest classifier (multiclass) but doesn't seem to be a recognised scorer for the scoring parameter. I do ...
0
votes
2answers
80 views

How to professionally spell library names such as "scikit learn"?

For example for inclusion in a CV. This is mostly a question of whether one should use capital letters or not. Usually I search the library to find how the library name is spelled on the official page....
1
vote
2answers
54 views

What is the difference between LabelBinarizer and MultiLabelBinarizer?

I am trying to understand the difference between the two label encoding techniques for output variable. I have read things but still can't get a clear picture as what makes them different. Also can we ...

1
2 3 4 5
40