Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Cannot impute 1D array

I'm trying to impute 1D array with shape (14599,) with simple imputer with most_frequent strategy but it said it expected 2D array, i already tried reshaping it (-1,1) and (1,-1) but its error ...
0
votes
1answer
9 views

Simple Imputer cannot impute by column

I have X_train that shapes (14599, 13), i'm trying to impute NaN with column's median but somehow it imputes with row resulting error because in a row there are date, and other than integer values. I ...
-1
votes
0answers
22 views

Reference pipelines/solutions for machine learning tasks [closed]

I am looking for a collection of reference solutions to classification/regression datasets (preferably using the scikit-learn stack) for machine learning datasets, i.e. a representation of the ...
0
votes
1answer
27 views

Low memory error while performing degree 2 polynomial regression on (3000*1835) sized array

I am working on a problem to predict the revenue, a film will generate. Some of the features available in the data set are json collection for the crew, cast which worked in the film. I applied ...
0
votes
0answers
8 views

Can i use different types of encodings for categorical variables in one dataset

Should I mix encodings. For example for features age and income i have one type of encoding and for features typeOfPerson i have another?
1
vote
2answers
27 views

Incremental Learning with sklearn: warm_start, partial_fit(), fit()

I have built an ML model with the goal of making predictions for targets of the following week. In general, new data will come in and be processed at the end of each week and be in the same data ...
4
votes
2answers
221 views

Cross Validation - Why does more folds increase variation?

Can someone explain why increasing the number of folds in a cross validation increases the variation (or the standard deviation) of the scores in each fold. I've logged the data below. I'm working on ...
1
vote
0answers
16 views

Retaining past learning with Incremental Learning [closed]

I have built an ML model with the goal of making predictions for targets of the following week. In general, new data will come in and be processed at the end of each week and be in the same data ...
2
votes
1answer
31 views

Adding anomalies to the Dataset

Recently I have been trying different Scikit-Learn anomaly detection clustering methods, like DBSCAN Isolation Forest. Based on how many training data I use, how I tweak on the algorithms ...
-1
votes
1answer
24 views

How to use entire file as unique sample for classification in scikit-learn?

My dataset is split in different files, since i'm using EEG data collected for BCI (Brain-Computer Interface) classification. Here is what i have: Each .txt file ...
0
votes
0answers
23 views

Online courses for Anomaly Detection

As the title say I have been looking for some online courses that would teach me about anomaly detection using Unsupervised Machine Learning. I want to focus only on Scikit-Learn and not go deeper ...
1
vote
0answers
15 views

Multi-label classification yielding too much unlabeled rows

I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
2
votes
0answers
15 views

Logic Check: Building a SKLearn Pipeline

I am new to the concept of building a pipeline in SKLearn and would appreciate some sense-checking to ensure that I am not leaking info from my training sets into my test set. Background: I have a ...
2
votes
1answer
18 views

What is the proper way to use time-series data for classification?

I have a time-series dataset for a classification problem. The data contains brain signals collected via EEG eletrodes along 2 seconds in frequency (Hz). The classes are divided in different files (so ...
2
votes
1answer
28 views

Overfitting with sklearn pipeline - reasons why?

So.... I've been playing around with this for FAR TOOO LONG now and I really need some advice. Most people on kaggle concat training and testing set TOGETHER and then pre scale the data, this seems ...
0
votes
1answer
10 views

Multi-class clasification

Just getting my toes wet with running some models on the Iris dataset and was wondering if using One-vs-Rest is required or not? Because I can fit a linear model without it, but using OVR yields ...
0
votes
0answers
13 views

Why is it giving me error of “Expected 2D array, got 1D array instead” [migrated]

I used regressor.fit([X_train], [Y_train]), it did worked but when I ran the below code ,it gave me the following error "ValueError: shapes (1,9) and (21,21) not aligned: 9 (dim 1) != 21 (dim 0)" ...
1
vote
2answers
31 views

Sklean pipeline order

Is there a correct order I should put data transformations into a pipeline using Sklearn? Currently I have these items in my pipeline; Feature selection, skew removal, scaling, outlier removal, ...
0
votes
1answer
16 views

Potential speedup by applying PCA once on dataset with m rows vs. IncrementalPCA to x batches of size m/x?

I've been working on trying to perform dimensionality reduction on high-dimensional, high-volume datasets (with many rows and columns - around 100,000 - 1M rows and ~500 columns). While the size of ...
2
votes
2answers
81 views

Extract relevant features from time series data

I have a time series data set from a sensor and the task is to predict the time before a failure event is occurred. The data set has one feature and has almost 20 million rows. This is a regression ...
2
votes
1answer
21 views

How can you adjust a prediction based on features in the future being different than predicted?

I have a model that takes mostly cumulative data, and makes a prediction. It's not baseball, but I'm using this as a pretty accurate analogy. You put in all the totals so far, and it make a prediction ...
0
votes
0answers
19 views

Variable selection involving mixture of numerical, high cardinal,low cardinal features

Consider a dummy dataframe: A B C D …. Z 1 2 as we 2 2 4 qq rr 5 4 5 tz rc 9 This dataframe has 25 independent variables and one target variable ,the ...
0
votes
0answers
17 views

Is there a function that could find slope of a curve ignoring peaks?

Let's say I obtained a timeserie (in blue) (with some missing data) that is (as far as i understood) : - following a general trend between specific points - more or less cyclic I have drawn the red ...
2
votes
2answers
25 views

predict an array like [1,2,3] increasing

I want to generate a simple model and classify it with decision tree. The idea is if numbers in an array are increasing then what I need is that. eg. ...
0
votes
2answers
53 views

Cannot make a single prediction: Is this behavior normal?

I am running a hate speech classifier published by Davidson et al. The principle is simple, the classifier takes as an input an annotated ('hateful', 'offensive', 'neither') dataset of tweets. It ...
1
vote
2answers
33 views

K-Means Clustering too crowded

I have written a simple python code that opens a csv files and then clusters the values of one column. There around 10k rows This is my code ...
1
vote
1answer
15 views

Elbow method on hundreds columns and rows

So I have these vectors called matrix_ after I applied TF-IDF (term frequency-inverse document frequency), and I also converted it to dataframe ...
3
votes
1answer
25 views

How to get feature importance from RandomForest using scikit-multilearn library?

I am working on multi-label classification problem, binary case. As a target variable there are five columns with 0-1 values. For a model training I use scikit-multilearn library. Below is my code ...
0
votes
0answers
29 views

How can I test my trained model on a completely new dataset? [duplicate]

Preface I have an annotated text dataset on hate speech. Simply put, the dataset consists of a column called text which includes a piece of text, and a column ...
2
votes
1answer
27 views

Is there a way to get the sample weights that were used to fit a scikit-learn estimator?

Many sklearn estimators support two weighting schemes: Per-class weights: given when creating the estimator object (e.g. ...
1
vote
2answers
35 views

Different performance for splitting into test/train data vs. using cross-validation

I am training a linear model using the following scikit-learn setup: ...
-1
votes
0answers
8 views

Use Python sklearn in Matlab, MLPRegressor

I would like to set up a MLP Neural Network in Matlab, using sklearn in Python. In Python the following code works fine: ...
1
vote
0answers
25 views

Stratified K fold same index present in both test and valid set

I am trying to do a stratified k fold cross validation for my dataset and want to keep an isolated 10% test set from the dataset and use the remaining for training and validation. Below is the code ...
0
votes
1answer
24 views

Where does spacy, nltk, prodigy, sklearn fit in the AI project?

Where does tools like spacy, sklearn, prodigy, nltk fit in the below AI project architecture and what are some common alternatives to these:
0
votes
1answer
24 views

How to train a classification algorithm with normalized data set using scikit-learn python

I have a dataset in CSV format, 6 columns and 1877 rows. The full dataset can be viewed at ShareCSV. The first five columns are characteristics and the final column is a binary result, I want to ...
0
votes
2answers
51 views

K-Means Clustering for data points with multiple attributes

I'm very new to K-Means clustering. Every example that I have seen has a two-dimensional data set. I am working to classify recipes of varying ingredient composition into families. Each recipe is ...
2
votes
2answers
56 views

How to split data into 3 parts in Python - training(70%), validation(15%) and test(15%) and each part have similar target rate?

I'm working on a company project which I will need to do data partition into 3 parts - Train, Validation, and Test(holdout). Does anyone know how I can split the data into 3 parts above and each ...
0
votes
0answers
19 views

Code freezes and never returns when linear_kernel (sklearn.metrics.pairwise) is used on 20M Movielens dataset

I'm fairly new to ML/AI, i'm trying learn the content based recommendation - here is my source code - https://github.com/jaganlal/content-based-recommender I'm using MovieLens 20M dataset - tags.csv ...
1
vote
1answer
15 views

Random Parameters to fix in ML to perform controlled experiments

Many algorithms and methods in modern Machine Learning techniques contain randomness, and because of that, running the same ML script several times can result in different outputs, therefore accuracy ...
0
votes
0answers
12 views

GridSearchCV and PyTorch with skorch shows error 'Invalid parameter lr' [duplicate]

I want to use sklearn's GridSearchCV in combination with PyTorch and use Skorch for compatibility. However, I receive an error telling me that ...
2
votes
0answers
15 views

GuassinaNB parital fit not working properly

I'm trying to make a partial fitting with GuassianNB here's small snippet of my code ...
0
votes
0answers
17 views

why we use “class_weight='balanced” as a classifier parameter?

Please I have imbalanced data and at first, I used SMOTE to oversample it, and it gives a good result, then I see some code used sometimes class_weight='balanced" inside of the classifier ( RL, DT, ...
0
votes
0answers
22 views

EarlyStopping in GridSearch - how to get the mean epoch after which training stopped?

is there a way to get the mean number of epochs when training stopped by EarlyStopping in GridSearch? ...
2
votes
0answers
18 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best hyperparameters ...
0
votes
0answers
34 views

ValueError while Plotting a specific column using K-Means

Following a tutorial, I was able to plot an array of data using KMeans. This is a csv file ...
3
votes
0answers
31 views

Data prediction using scikit-learn and a list

I have a group of lists detailing temperatures over differing amounts of time. My goal is to use machine learning to identify periods in which a machine is turned on and off, where turning on the ...
5
votes
1answer
51 views

How to pass custom distance functions to K nearest neighbors function in scikit-learn

I am trying to solve a problem where I am asked to perform classification using KNN but with a custom euclidean function: The function is the following: ...
4
votes
2answers
181 views

Compare Coefficients of Different Regression Models

in my project, I am using asuite of shallow and deep learning models in order to see which has the best performance on my data. However, in the pool of shallow machine learning models, I want to be ...
1
vote
0answers
41 views

How to predict using the pickle model?

I am new to data science and trying to learn something. I was able to complete the prediction with 98% accuracy and i saved it as pickle model. Now while trying to predict using this model I am ...
2
votes
1answer
37 views

Bagging or Random Subspace Method for Random Forest?

I am reading a lot about the random forest regressor. I reading about bagging (bootstrap and aggregation) and random subspace. But I am not sure if the random forest regressor just using bagging or ...

1 2 3 4 5 28