Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

Filter by
Sorted by
Tagged with
2
votes
1answer
14 views

variance threshold, returning the names of the selected features

I am trying the variance threshold method for the first time and I am following the example in sklearn to work on it. ...
0
votes
1answer
33 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
2
votes
0answers
19 views

interpolation - graphical quality evaluation

I try to compare different interpolation models quality and I'm looking for a graphical tool to do that. Application case: I'm not familiar with intepolation using neural networks. I decide to test it ...
2
votes
4answers
42 views

How to get model attributes in scikit learn (not hyper parameters)

How to get model attributes list (not hyper parameters passed to Estimator's class)? For ex: kmeans = KMeans(n_clusters=5) kmeans.fit(X) kmeans.labels_ how to ...
0
votes
0answers
20 views

Inverse predict the features from known target with fitted sklearn regressor

I understand that the default way a scikit-learn regressor works is that we fit it to a dataset of features and targets (X_train, ...
0
votes
0answers
15 views

Understanding Mapper clustering

Is there a way to find the number of clusters in mapper.map? It is a module in kmapper.KeplerMapper. When I plot the graph ...
0
votes
0answers
17 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
0
votes
0answers
7 views

Automatically find an optimal threshold for multiple datasets

I am running an unsupervised model on data with no labels to find outliers, and have 100 datasets in total. The way I do this is, first I run it using a default threshold on each dataset, and second, ...
1
vote
0answers
18 views

Decision Boundary with random new observations vs observations from test set

I'm trying to plot decision boundary for Decision Tree classifier. Classifier is trained on training set, and decision boundary (contour) using random new observations and observations from test set ...
1
vote
0answers
36 views

Using scikit-learn for classification problem where classifier based on high dimensionality data

I am using scikit-learn for time series price data for a spot market to categorize the time series points after as the same, higher, or lower price as the current time series point recorded. I am not ...
2
votes
1answer
63 views

Choosing a distance metric and measuring similarity

I am trying to decide which particular algorithm would be most appropriate for my use-case. I have dataset of about 1000 physical buildings in a city with feature space such as location, distance, ...
1
vote
0answers
13 views

Which version of spectral clustering in sklearn library?

Which version of spectral clustering is implemented in sklearn library? Is it Shi, Malik or Ng, Jordan, Weiss from this tutorial? In sklearn user guide, both versions are mentioned in reference. From ...
3
votes
1answer
20 views

Smaller alternatives to sklearn that doesn't require scipy?

I am packaging my model for deployment in aws lambda which has a size limit of 250mb for all dependencies. Sklearn, if you include its dependencies of numpy and scipy is a huge package. Are there any ...
0
votes
0answers
12 views

How to use a Multinomial Naive Bayes Classifier on different sets of data?

I am working on a sentiment analysis project involving tweets. I used a Kaggle dataset to train my model for sentiment analysis and want to use that trained model to predict the sentiment on an ...
0
votes
1answer
14 views

Significance of Number of Calls and Reset Call in Ball Tree

Why does the Scikit Implementation has functions to reset and get number of calls? How are these parameter important in Trees? https://scikit-learn.org/stable/modules/generated/sklearn.neighbors....
1
vote
1answer
59 views

What can weka do that python and sklearn can't?

I would like to build a variety of classification and regression decision trees. My use case focuses on extraction and communication of decision rules. Previously weka was used in my organisation for ...
2
votes
4answers
36 views

Scikit-learn: train/test split to include have same representation of two different types of values in a column

I have a dataset of online purchase orders that contains two types of customers: Customers who have an account and thus are known customers with unique customer number. Customers who do not have an ...
4
votes
2answers
70 views

The impact of using different scaling strategy with Clustering

I'm currently learning about clustering. To practice clustering, I am using this dataset. After running K-means clustering for multiple values of k and plotting the results, I can see that scaling is ...
0
votes
1answer
39 views

Does stronger regularization always improve performance on testing set?

I am using the Sklearn logistic regression function to do a binary classification task on texts. I did the task using three different inputs: Bag-Of-Words, TF-IDF, Doc2vec embeddings. The question is ...
1
vote
0answers
15 views

Is SKLearn's KS Test Implementation Wrong?

I was looking at the implementation for the KS TEST in sklearn here: https://github.com/scipy/scipy/blob/v1.5.0/scipy/stats/stats.py#L6580-L6745 ...
2
votes
1answer
50 views

Choose threshold to get 90% precision classifier - ML Binary Classification problem

I have chosen threshold value with below code to get 90% precision classifier ...
0
votes
0answers
10 views

how to improve score of automl regressor

I was trying to solve a regression problem on HackerEarth. I got to a score of 81.20 using XGBRegressor after some data preprocessing, the top ranker had a score of 81.55. Some one suggested me to use ...
1
vote
0answers
16 views

AdaBoost decision_function() outputs in binary classification with sklearn

As I understand it based on some study of the source code, I would expect, when using AdaBoost, that values obtained by calling decision_function() would be bounded between -1 and 1. This is because ...
1
vote
2answers
17 views

Biasing SVM algorithm towards particular subset of data

I'm training an SVM model for sentiment analysis, based on social media data eg. tweets. The model will be trained using a small selection of a particular company's tweets in order to classify new ...
0
votes
1answer
34 views

Random forest confusion matrix encountered invalid values

I am doing classificaion using random forest classifier in python (scikit learn). I have many different databases, each one has 33 observations and the prediction is based on 600 columns. The script ...
0
votes
1answer
124 views

Error with pandas dataframe (needs to be 1-dimensional)

I am trying to determine the conformal predictions for my model with my data. But it gives me following error that occurs at icp.calibrate(X_cal, y_cal) : ...
0
votes
1answer
39 views

Why PyTorch is faster than sklearn models?

Recently, I get to know about the hummingbird library for Python. I trained a RandomForest on a 10M-sized dataset with 2 labels. With sklearn it was taking 450 ms for inference. But after converting ...
0
votes
1answer
31 views

what is difference between fit and fit_transform in sklearn while applying feature scaling [duplicate]

I have seen few post related to this question but i am not quite clear about my confusions as mention bellow. I have some confusion related to fit and fit_transform. suppose, I have X_train and X_test ...
0
votes
1answer
21 views

Multidimensional K-Means wiith sklearn, centroids problem when plotting

I am working with a dataset (X) to predict 12 clusters with K-Means using python SKLEARN library: ...
1
vote
1answer
32 views

How to set different weights for different training samples?

Suppose that your supervised learning training set is made out of 3 different datasets, merged into a big one. Because of the way each of those was labeled before merging, you might be suspicious that ...
2
votes
1answer
25 views

Overfitting results with Random Forest Regression

I have one image that contains for each pixel 4 different values. I have used RF in order to see if I can predict the 4th value based on the other 3 values of each pixel. for that I have used python ...
0
votes
3answers
49 views

How do i get dummies for this dataset

I am using an udemy course for MachineLearning and I am trying to form a dummy for my variable the column is Country I want to change to France Germany Spain France ...
0
votes
1answer
14 views

How do feature selection on a sparse matrix?

Say I want to do features selection on a sparse matrix, i.e., 10,000 rows x 1500 features, but the matrix is mostly sparse. Let's say the features are all numeric and the target is binary and discrete....
0
votes
3answers
119 views

GridSearchCV with Random Forest Classifier

I'm working with a supervised learning problem and trying to predict a binary label and using a Random Forest to do so. I'm trying to tune my hyper-parameters to give me a best model based on my data. ...
4
votes
3answers
261 views

Should I use keras or sklearn for PCA?

Recentl I saw that there is some basic overlapping of functionality between keras and sklearn regarding data preprocessing. So I am a bit confused that whether should I introduce a dependency on ...
1
vote
1answer
47 views

Interpretation of scikit-learn one class svm scores

How can I interpret the scores generated by the function score_samples(X) from a scikit-learn OneClassSVM model? Is there a way ...
4
votes
1answer
121 views

What happens if at leaf node both classes have same number of samples?

I analyzed a small dataset which had three features, so I kept max_depth of decision tree to be 3, in doing so I found it something intresting, there was a leaf node which had number of samples of ...
1
vote
1answer
130 views

Lightgbm confidence interval

I want to compute a confidence interval for each sample for a lightgbm model I've trained. If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
1
vote
1answer
18 views

How to enable GPU on GradientBoostingClassifier?

Is there a way to enable GPU on GradientBoostingClassifier?
4
votes
2answers
95 views

ExtraTreesRegressor criterion

As I understand, ExtraTreesRegressor from sklearn works by doing random splits instead of minimizing a metric like gini for ...
0
votes
1answer
20 views

Clustering with k-means for text classification based on similarity

I have a column that contains all texts that I would like to cluster in order to find some patterns/similarity among each other. ...
0
votes
1answer
12 views

How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only

I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
0
votes
1answer
12 views

Can we calculate AUC for deep learning based regression task

There is a paper [ref.attaced], where they used a deep learning based regression and evaluated using mse and AUC. The targets are continuous values such as 1,2,3 up to 16 and has been normalized ...
-1
votes
1answer
20 views

How do I view my K features with SelectKBest? [closed]

I am getting an error for "no attribute columns". Because of this, I can't see the selected K features and can't build plots: First occurrence of error: Second occurrence of error: I'm also ...
0
votes
0answers
11 views

Using scikit-learn instead of mlr — what would the code look like?

So I would like to test out scikit-learn and would like to know how the code below (using mlr) would be written in scikit-learn. ...
0
votes
0answers
13 views

Getting error ValueError: Found input variables with inconsistent numbers of samples: [6, 21]

I'm trying to evaluate the perfomance of my algorith by getting the Mean absolute error, mean squared error and root mean squared error. But my output is value error ValueError: Found input variables ...
0
votes
1answer
16 views

Text Classification on a very small data set with a lot of classes

I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below. ...
0
votes
0answers
8 views

Neural network - calibration strategy & cross-validation

I have a hard time articulating the different parts of a calibration process of a relatively vanilla neural network. I am mostly concerned with : Grid search for the regularisation hyperparameter ...
1
vote
1answer
30 views

How to use ColumnTransformer and FunctionTransformer to apply the same function to many columns, but separately?

I want to apply pd.cut as a transformer in a pipeline, like this: ...

1
2
3 4 5
32