Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
0
votes
0answers
27 views

The Difference between One Hot Encoding and LabelEncoder? [duplicate]

I am working on a ML problem to predict house prices and Zip Code is one feature which will be useful. I am also trying to use ...
0
votes
1answer
204 views

Using a pipeline and transforming data with imputing and OneHotEncoding performs worse than get_dummies

I'm still in the process of learning, so I'm sorry if this doesn't make much sense. I'm doing Kaggle learns micro courses, and to work with missing tabular data we learned about using pipelines with ...
0
votes
0answers
18 views

How do I organize a multi-site multivariate time-series dataset for a Random Forest Regression?

I am trying to do a Random Forest Regression to forecast the next months value. I have a few years of data split by month. In each month I have about 1500 unique sites. There are 14 features.
0
votes
0answers
79 views

How to gauge overfit with MLPClassifier and cross_val_score?

I'm learning sklearn. When using MLPClassifier.fit() and MLPClassifier.predict() I would ...
0
votes
1answer
57 views

Splitting large multi class dataset using leave one out scheme into train and test

I am doing some supervised learning using neural networks, and i have a Targets array containing 1906 samples, which contain 664 unique values. min. count of each unique value==2, by design. Is there ...
0
votes
0answers
83 views

How to detect anomalies (errors and exceptions) in log files?

Is this a good approach? So I'm working on a Root Cause Analysis system which should help find the cause/the root error of failed system builds (packaged in a tarball), through the analysis of log ...
0
votes
0answers
34 views

do I need to upload all model files?

here is the link of the model inside the link there is a biobert model and i do not understand that how can i implement this model on a simple html page where there is a textfield for the model input? ...
0
votes
0answers
26 views

How to find coeffcients from pre-trained model

I have trained and pickled logistic model. Have also successfully loaded the pre-trained model and performed prediction on unseen/new data. However, would like to know how to find coefficients/...
0
votes
0answers
92 views

Cluster based on both positions and similarity scores

I have a dataframe position giving me the x and y positions of 87 points. I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
0
votes
0answers
112 views

Regarding sklearn adjusted_rand_score in k means clustering

A dataset is given consisting of target class as categorical.K means was applied to it for clustering the data and got the corresponding cluster labels. But how to put the values in adjusted Rand ...
0
votes
0answers
10 views

How to find anomalies using NMF

I am new to NMF(Non negative matrix), I am trying to use it to detect anomalies in my credit card data (unsupervised data) However after generating the components that is nmf.components_ I am not ...
0
votes
2answers
181 views

How to train model to predict 1 value from multiple input samples

Very new in ML... Actual problem is more complex, but I'll give it shorter. Train data is a collection of samples, describing features of some items. There are many samples for each item. So ML ...
0
votes
1answer
70 views

metric accuracy for linear regression and non linear regression

Is there a one size fit all metric to measure accuracy / error rate for both linear or non linear regression models? For example adjust R2 is only for multi linear regression (or so they say). RMSE ...
0
votes
0answers
12 views

Classifying Short Texts with Spatial Features

I have a dataset of short texts (like tweets) in addition there's some geographical data attached to each tweet - coordinates, whether it was made on the road, street, outside or in the building, ...
0
votes
1answer
79 views

adding a supervizing process during knn process

I try to improve my knn regression process (I use sklearn / python, but it doesn't matter). Because I can have a scientific point of view on my data, I would like to improve my results. I give you an ...
0
votes
3answers
744 views

how to avoid tokenizing w/ sklearn feature extraction

I'm trying to analyze some machine log files and the column I'm looking at can have values like 'Part.C1.11.Reading Status'. I want to treat the complete string as one token and I don't want it to be ...
0
votes
0answers
9 views

Custom Imputation relative to targets in TRAIN and TEST sets

I have a methodology question for dealing with heaps of missing data in my project. My dataset is composed of parts A (~200 columns) and B (another ~200 columns). Together they are to be used for ...
0
votes
2answers
26 views

How do I fix mis-rendered matplotlib?

How do I correct my data or format it so that it is presentable, and fix my graphs? Dataset is 345551 rows × 7 columns. I am using numpy, pandas, seaborn and matplot lib. It seems that my pricing ...
0
votes
2answers
230 views

What is the best way to replace NaN values for ranked columns

I have a column named BsmntQual that gives a ranking on the height of the basement per each house. These are all of the unique values in this column: ...
0
votes
0answers
40 views

Tag words of interest by using machine learning?

I have a set of documents (around 50k), each document is a sentence long. I would like to tag each document with the words that relate to risks. For example, "dangerous", "hazard", "fatal", etc... If ...
0
votes
3answers
24 views

How to find the driver features towards a particular result in Classification problems

In a classifier model, we can predict the outcome class, but here I need to find out the features that drive towards a particular result in a classification problem, that are a strong indicator of a ...
0
votes
0answers
48 views

Passing multiple rows as a single batch for prediction using sklearn

I am trying to train a standard sklearn ml model (random forest). However, my data is a collection of rows, each having a column of date and ...
0
votes
0answers
16 views

PCA giving separated results expected (jupyter sklearn)?

I'm a complete newbie to PCA and I have 3 sets of values which I want to plot with PCA. This is what I am using: ...
0
votes
0answers
118 views

Pre-processing data to make predictions on deployed Sklearn model

I am new to Machine Learning. I have trained a ML model on the Diamond Prices Dataset to predict the price of a diamond given it's features (carat, cut color, clarity, etc...) I have used pickle to ...
0
votes
1answer
298 views

Accuracy of machine learning models

I started learning ML and I have some problems with evaluating / finding the accuracy of regression and classification models. Till now I used .score() in both ...
0
votes
2answers
48 views

Principal component analysis

I have a data set that looks like the following: ...
0
votes
0answers
526 views

Why would one use entropy instead of Gini index in CART?

I read this question Gini Impurity vs Entropy and was wondering why would someone use entropy instead of Gini index in a decision tree with scikit-learn. Indeed, I find these arguments legit: ...
0
votes
1answer
30 views

How to tune parameters batch by batch?

As the title states, I am trying to cluster a huge dataset and cluster it by using sklearn.Birch to learn incrementally. If it's a small dataset, I could just use ...
0
votes
0answers
37 views

Can one set manual adaptive learning in SGDRegressor()?

I wanted to update learning rate $r = r/2$ in each iteration of SGDRegressor(). I cannot find any way so far to update the learning rate manually. There is a choice called adaptive but it doesn't look ...
0
votes
0answers
40 views

How to reduce / avoid false predictions with sklearn and MultinomialNB?

I'm using sklearn to predict product groups from product titles. That is working very well, if the titles are similar to the ones I labeled. Simplified example: ...
0
votes
0answers
34 views

Why the accuracy decreased with more data

When working with a random forest model, it is surprising that the cross-validation score on the training dataset is much higher than the score on the whole dataset. Here is the code: ...
0
votes
0answers
25 views

Cannot explain Naive Bayes prediction on toy data

I tested Naive Bayes from sklearn on the toy data from Tom Mitchell's book Machine Learning. The results are unexpected. The very first instance should be classified as "No" according to the ...
0
votes
0answers
53 views

Unsupervised learning/ clustering for data with multiple categorical variables

Dataset: I have been trying unsupervised clustering algorithms (K-modes & SOM) to cluster the students based on their grades in 3 exams. Should I one-hot encode the data (even though grades are ...
0
votes
0answers
52 views

Building recommendation engine from transactions data

I am trying to build a recommendation engine for an e-commerce company and I have the following input files : 1) past user transactions + in-app events 2) a new list of campaigns I should recommend ...
0
votes
0answers
27 views

How to use contrast coding schemes in the case of multiclass target variable? How to encode categorical features if contrast coding fails?

How do you deal with a dataset which only has categorical variables, all of whom have high cardinality? What is the right way to encode high cardinality categorical variables if the target variable ...
0
votes
0answers
67 views

How to compute AUC in gridsearchCV for multiclass problem

I'm currently working on a multiclass imbalanced problem. I am using random forest as learner and using different methods of resampling. I would like to use ...
0
votes
0answers
61 views

Has anybody used alternative hyperparameter optimization techniques (other than default one) in SK-Learn?

I've been using Sklearn for Gaussian process regression that has L-BFGS-B (“fmin_l_bfgs_b”) as a default optimization algorithm. I want to implement some other ...
0
votes
0answers
53 views

Understanding `get_combination_wise_output_matrix` when investigation a multi-label classification problem

I am currently working on a multi-label classification problem. I am using the scikit-multilearn library (further reading here) I understand that train / test split is important for these types of ...
0
votes
0answers
92 views

train_test_split() error: ValueError: Found input variables with inconsistent numbers of samples: [10015, 7]

I get this for x.shape and y.shape (7, 1, 10015) (10015, 28, 28, 1) How can I get the train test to work? ...
0
votes
0answers
12 views

Dict with features and classification -> two seperate but aligned lists

For my ML project, feature set extraction is expensive, but I need to be able to retrain the model on a fairly regular basis. For this reason I'm extracting my features from all of my documents at ...
0
votes
0answers
34 views

Variable Importance in sklearn's RandomSearchCV

I ran an algorithm for fitting with RandomSearchCV on a classification problem, the results were good but I need to know which variable had more importance. I know that a fitting without ...
0
votes
0answers
74 views

Using logistic Regression with multi_class='multinomial' for binary classification

I have built a binary classifier and I have tested the results with multi_class='multinomial' and multi_class='ovr'. The results are relatively better using multinomial option. I understand that ...
0
votes
0answers
25 views

Is the precision_recall_curve reasonable?

I always get this 'pattern' PR curve when I use precision_recall_curve function to plot PR curve. The starting point is always from (0,1). Because when thresholds set up the highest score lend to ...
0
votes
0answers
37 views

How to fit a Gaussian Process with a product kernel K(x,a)=K_1(x,x')K_2(a,a') with scikit-learn?

I have a training dataset in the form of $(x, a, y)$ where $x$ and $a$ are two arrays of features and $y$ is the target outcome. I am interested in fitting a Gaussian Process with a product kernel $...
0
votes
0answers
65 views

While using keras MLPRegressor, the loss value does not change at all

Days ago I use sklearn MLPRegressor and build a model, loss value begins at around 26 and keeps decreasing. However, when I use keras and build a same NN, the value does not change. Pictures above is ...
0
votes
0answers
28 views

ML with imbalanced binary dataset

I have a problem I am trying to solve: - imbalanced dataset with 2 classes - one class dwarfs the other one (923 vs 38) - f1_macro score when the dataset is used as-is to train RandomForestClassifier ...
0
votes
1answer
64 views

Plotting Polynomial Regression?

I'm reading through Hands-On Machine Learning with Scikit-learn and Tensorflow by Geron. I am creating a simple polynomial regression using sklearn's ...
0
votes
0answers
129 views

How to implement custom early stopping metrics in sklearn MLPClassifier?

I am trying to use MLPClassifier as a predictor using the predict_proba method. Unfortunately, the standard early stopping function is not working for me. Is there a way how to implement a custom ...
0
votes
0answers
29 views

SVM classifer gives the same output for all test cases

I have a dataset belonging to seven different classes. Each input consists of 1024 features. There are only 30 samples belonging to each class available for training. I have used several ...
0
votes
0answers
58 views

How to calculate the ranking loss in multilabel classification?

I can calculate the ranking loss using sklearn. But I am not able to understand this calculation manually in a step by step process. Can anyone explain? ...