Questions tagged [scikit-learn]

scikit-learn is a popular machine learning package for Python that has simple and efficient tools for predictive data analysis. Topics include classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.

Filter by
Sorted by
Tagged with
1
vote
2answers
85 views

How can the labels of AgglomerativeClustering be re-computed?

I'm using scikit learn's AgglomerativeClustering on a large data set. I want to modify the distance_threshold after the model has already been computed. Computing ...
2
votes
0answers
15 views

Methods of de-emphasizing some dimensions in a cluster analysis

I'm trying to understand how "weightings" on different dimensions in a cluster analysis might relate to the range of values along a given dimension in the dataset. DATA SET List of 1,000 to ...
0
votes
1answer
56 views

IterativeImputer Evaluation

I am having a hard time evaluating my model of imputation. I used an iterative imputer model to fill in the missing values in all four columns. For the model on the iterative imputer, I am using a ...
1
vote
1answer
35 views

Geolocation Based Anomaly Detection in IPs Using Isolation Forest

I'm trying to detect anomalies based on geolocation from IP addresses on a server access log file. I have created two features country and geo_velocity, using the IP address and the timestamp of each ...
-2
votes
3answers
52 views

How to decrease $R^2$ value and change it to positive value [closed]

I'm working on a data, and use regression , as you see bellow: from sklearn.svm import SVR regressor = SVR(kernel = 'linear') regressor.fit(trainX,trainY) above ...
0
votes
1answer
758 views

How to use inverse_transform in MinMaxScaler for pred answer in a matrix

I am working on a data, for preding output, I used SVR by bellow code: ...
1
vote
1answer
41 views

Kernel Density in Scikit Learn

I'm trying to understand how does the KernelDensity class in scikit-learn work. Consider the following two cases which build a kernel from two different arrays (a). I'm wondering why the result of ...
1
vote
1answer
54 views

Accuracy over different sample sizes from dataset

What I'm trying to do is predict how much more data would help in a classification task. So, what I'm doing is bootstrapping entries in my dataset to get a sample, with a specified size. Then, I fine-...
1
vote
1answer
28 views

Confidence score for all observations is between 0.50 - 0.55

Hello Data Science Stack Exchange Community, This question will appear to be open-ended, however any answers or thought will be much appreciated. I am trying to go-through a pre-trained Random Model ...
0
votes
1answer
41 views

GridSearch on imbalanced datasets

Im trying to use gridsearch to find the best parameter for my model. Knowing that I have to implement nearmiss undersampling method while doing cross validation, should I fit my gridsearch on my ...
1
vote
1answer
44 views

Perform PCA on columns of different length

I have about 20-30 columns, all with different lengths. The first column has 25000 rows, the second column has 19000 rows, and it's different for all the columns. All are the survey data with 0 (No),1(...
0
votes
0answers
46 views

Implementing Connectivity: “IndexError: too many indices for array: array is 3-dimensional, but 4 were indexed”

I am trying to implement connectivity as a feature within my code, but am unsure of how to fix this error code. Here is my code up until the point of the error. ...
0
votes
0answers
13 views

How to train a text classifier for product search query to determine category

I am trying to train a product search query (e-commerce) classifier for deducing probable product categories from search query with a dataset of 700k queries with probable categories labelled I tried ...
0
votes
0answers
26 views

How to make a Label Encoder trained on a training dataset transform an unseen value of a test dataset?

During the data preprocessing stage, I decided to apply the Label Encoding on one of the columns because it contained data points in string format. Suppose the column contains the following distinct ...
0
votes
0answers
19 views

Cross validation with GridSearchCV or train-val-test split

I have a question regarding the CV in GridSearchCV. To test my model should I split my data into 3: training, validation, test? For easy understanding let's say my data is split into training with 60% ...
0
votes
0answers
28 views

How can I compare imputation techniques on a dataset with sci-kit learn?

I have a dataset data that has missing values. I am trying two ways of imputing these values, but I would like to compare them. In the first method I am using a ...
1
vote
1answer
39 views

Getting very low/ wrong accuracy from RandomizedSearchCV

I am currently using RandomizedSearchCV to optimize my hyper-parameters. However the reported scores of each iteration is very low. When I then evaluate the highest scoring candidate I get very high ...
0
votes
1answer
14 views

Treating continuous data as a classification problem by predicting bins or quintiles

I currently have a model that has several numeric Y or predicted variables Sample Data: Y1 Y2 ... YN 2710 0.32 ... 31231 1710 0.52 ... 51231 I am currently using regression (multioutput regression ...
1
vote
0answers
22 views

How to use sklearn's Matrix factorization to predict new users' scores

I'm trying to use sklearn.decomposition.NMF to a matrix R that contains data on how users rated items to predict user ratings ...
-1
votes
1answer
18 views

Increase accuracy in binary classification with ambiguous data

I am fairly new to Datascience and currently working on an assignment that requires me to do a binary classification on a set with about 9 parameters for X. I tried working on it using different ...
0
votes
1answer
18 views

Math of Logistic regression cost function

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function: $$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^...
3
votes
2answers
115 views

Preprocessing: StandardScaler() Do we really need mean to be zero?

For instance, many elements used in the objective function of a learning algorithm (such as the RBF kernel of Support Vector Machines or the l1 and l2 regularizers of linear models) assume that all ...
1
vote
1answer
18 views

Why is the numeric column treated as a categorical column in Microsoft Learning?

Learning Microsoft's DP-100 and saw this notebook: The part of its code which I have question with is ...
0
votes
1answer
111 views

How to remove features from a sklearn pipeline after it has already been fitted?

Background: I have created a basic modeling workflow in sklearn that utilizes sklearn's pipeline object. There are some preprocessing steps within the pipeline, and the last step of the pipeline is to ...
1
vote
0answers
51 views

Can you estimate average precision from log loss?

I am doing my final thesis in the field of Deepfakes and their detection. The final outcome is to have a binary classifier which could predict which video was updated and which was not. In other words,...
0
votes
0answers
25 views

Using nearest neighbour as mapper of xy coordinates

This is my first post here so I apologize if this is not right place for this kind of question. I am looking for some tips on using (k)nearest neighbor algorithm as a mapper of hypothetical position ...
1
vote
1answer
34 views

pairwise_distances with Cosine and weighting

Is there a way to get a weight into the pairwise_distances(X, metric='cosine') Potentially using **kwrds? ...
0
votes
0answers
32 views

Why does normalization improve my decision tree performances?

I have a regression problem for which I have to try several models, so I normalized my data and then tried to use a decision tree regressor (from sklearn.tree) and I noticed very good results (...
0
votes
0answers
22 views

How do I find the eigenvectors corresponding to the largest eigenvalue of a matrix in scikit?

Im trying to determine the principal component 1 and 2 of a symmetric matrix using sklearn. Id appreciate any help. Thank you.
-1
votes
1answer
26 views

Printing the tweets that were incorrectly predicted after applying a machine learning classifier

I applied the random forest classifier to my csv file to classify the tweets as spam or not spam and after an accuracy of 93%, when I printed the confusion matrix I got [[1068 105] [ 65 1262]]. Now ...
1
vote
1answer
98 views

What does the oob decision function mean in random forest, how get class predictions from it, and calculating oob for unbalanced samples

I am interested in finding the OOB score for random forest using sklearn, when it is used for a binary classification task, and there are unbalanced samples. What does the oob decision function mean ...
0
votes
1answer
213 views

What is the difference between CountVectorizer() and Tokenizer() or are they the same?

from sklearn.feature_extraction.text import CountVectorizer from keras.preprocessing.text import Tokenizer I am going through some NLP tutorials and realised that ...
2
votes
1answer
74 views

Multi-Feature One-Hot-Encoder with varying amount of feature instances

Let's assume we have data instances like this: ...
0
votes
2answers
37 views

Random Forest Classifier cannot recognise parameter grid

I am trying to run the below code to extract the feature importances of my random forest, but I'm getting the following error TypeError: init() got an unexpected keyword argument '...
1
vote
0answers
15 views

KNN with high-variance data [closed]

KNN doesn't work well with high-variance data, so how should I fit my data? Here is an example of what the data looks like:
1
vote
1answer
62 views

How to combine two logistic regression models trained on different set of data?

My data has a hierarchy structure - meaning that there is an N class at level 1 and an M class at level M. After training both models separately with a different set of data (both are Logistic ...
1
vote
0answers
23 views

How do I convert strings in a dataframe column to int or float [closed]

Encoding the column seperately works ,but when I try it on the dataset directly it throws an error. The error is something like this. ...
1
vote
0answers
18 views

How to compare hyperparameter tuning in R and Python

I tried random forest in both R (Caret) and Python (Scikit-learn), but the results differ drastically. Pearson correlation between predicted value and actual value was 0.2 in python whereas 0.8 in R. ...
0
votes
0answers
36 views

How to predict future prices with Keras LSTM time-series prediction model?

I have a trained and tested LSTM model which is meant to predict Ethereum close prices using all time csv data (24h steps). How do I now go about inputting an empty dataframe with future dates to ...
1
vote
0answers
77 views

Can elastic net l1 ratio be greater than 1?

I have multiple datasets that I trained with ElasticNetCV (sklearn), and I noticed that many of them selected l1_ratio = 1 as ...
2
votes
1answer
28 views

Large dataset - ANN

I am trying to classify around 400K data with 13 attributes. I have used python sklearn's SVM package, but it didn't work, and then I learned that SVM's are not suitable for large dataset ...
-1
votes
1answer
23 views

Kaggle Titanic Problem

I have two datasets train and test I did all the data cleaning on both the datasets.In my test dataset I don't have the dependant variable while in the train set I have dependent as well as ...
0
votes
0answers
24 views

Predicting algorithm elapsed time using a Gaussian Process

I am new to the Gaussian Processes. I have a simple problem: given some training data of an algorithm performance given its problem size, I want to predict how long it will take (in seconds) to ...
1
vote
1answer
42 views

How to impute missing value in Test Set using a custom Imputer created on training dataset

I am working on a toy project to predict claims. One of the input features has null values on which I have applied a custom imputation technique. Under this technique, I replaced missing values with ...
0
votes
0answers
63 views

Impute missing values in feature column on the basis of Target column

I am working on a toy project for insurance claim prediction. In the input data for one of the feature (numeric data type) half of the values are missing. My target variable is binary which indicates ...
0
votes
1answer
95 views

How to interpret my logistic regression result with statsmodels

so I'am doing a logistic regression with statsmodels and sklearn. My result confuses me a bit. I used a ...
1
vote
0answers
90 views

How to interpret my logistic regression result?

I'm having a hard time to interpret my result of the logistic regression. I have a few question. Firstly, how can I check if a feature is more important to the others, like that there is a real ...
-1
votes
1answer
25 views

Is it mandatory to change the dtype='object' to 'category' before label encoding [closed]

I have seen some people change the datatype(from object to category) of the feature they want to encode.
3
votes
1answer
25 views

Selecting a boundary on a binary classifier to optimal precision and recall

I have a logistic regression classifier that shows differing levels of performance for precision and recall at different probability boundaries as follows: The default threshold for the classifier to ...
1
vote
0answers
67 views

Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...

1 2
3
4 5
37