Questions tagged [scikit-learn]
scikit-learn is a popular machine learning package for Python that has simple and efficient tools for predictive data analysis. Topics include classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.
2,299
questions
1
vote
2
answers
347
views
Plotting Polynomial Regression?
I'm reading through Hands-On Machine Learning with Scikit-learn and Tensorflow by Geron. I am creating a simple polynomial regression using sklearn's ...
CommunityBot
- 1
1
vote
1
answer
155
views
+50
How to solve a non-linear system with the GAUSS-NEWTON algorithm in Python? (Jacobian matrix J, etc.)
I would like to solve a non-linear system (which contains the goals of a football team in previous matches) using the Gauss-Netwon algorithm, in order to find the parameter (of frequency) to use as ...
- 61
0
votes
1
answer
151
views
Text Classification misclassifying?
I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0).
My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
CommunityBot
- 1
0
votes
1
answer
67
views
How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only
I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
CommunityBot
- 1
0
votes
1
answer
278
views
How to save multi-output predicted masks into two different folders after using model.predict_generator
I have a multi output segmentation task, the training process went well, but when Im trying to get the prediction I found difficulties to separate the two output into two different folders, in my ...
CommunityBot
- 1
0
votes
1
answer
84
views
How to prune the features using recursive feature elimination?
I have some kind of spatial data for nearly 1000 locations and at each location around 5000 features. I am doing neighborhood analysis to identify which features are predominant in local neighborhood.
...
CommunityBot
- 1
0
votes
1
answer
15
views
Number of stop words variation in libaries sklearn and nltk
Is there a reason why there is a big variation in the number of stop words? I assumed that there would be a general agreement from English experts how many stop words there could be. And even with ...
- 418
0
votes
1
answer
9
views
Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?
I have been working with Python for machine learning and have a fair amount of code written in Python using libraries such as scikit-learn, pandas, and numpy. Recently, I’ve been faced with larger ...
1
vote
2
answers
467
views
Sklearn LocalOutlierFactor contamination parameter
Can anyone provide an intuitive explanation of the choice of contamination parameter used in sklearn's LocalOutlierFactor implementation when ...
CommunityBot
- 1
1
vote
3
answers
6k
views
GridSearchCV with Random Forest Classifier
I'm working with a supervised learning problem and trying to predict a binary label and using a Random Forest to do so.
I'm trying to tune my hyper-parameters to give me a best model based on my data.
...
CommunityBot
- 1
1
vote
1
answer
20
views
Out-of-Range Target Variable in Sequence-based Machine Learning Model
I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the ...
- 418
1
vote
1
answer
79
views
Is there a clustering algorithm that can cluster time series dataset based on variation ratio (or quantity)?
I am learning machine learning from scikit-learn and reading its docs.
Clustering clusters groups based on the Euclidean distance and filters them by different ways ex: Gaussian distribution, or mean-...
CommunityBot
- 1
0
votes
1
answer
822
views
How do feature selection on a sparse matrix?
Say I want to do features selection on a sparse matrix, i.e., 10,000 rows x 1500 features, but the matrix is mostly sparse. Let's say the features are all numeric and the target is binary and discrete....
CommunityBot
- 1
1
vote
2
answers
88
views
How to get different results running sklearn's MeanShift in a single program? (Python3)
I ran into a quirk with sklearn's MeanShift that I don't know how to get around.
MeanShift doesn't predictably give the same results on every run, so I wanted to run it multiple times within one ...
CommunityBot
- 1
0
votes
1
answer
1k
views
sklearn predict: IndexingError: ('Too many indexers', 'occurred at index <name>')
The goal of what I'm trying to accomplish here is to have the output contain all of the use_cols but the model only be built to calculate on categorical_features. The output will then be used to ...
CommunityBot
- 1
0
votes
2
answers
543
views
Ways to increase recall in SVM
I am training an SVM on UCI's Bank Marketing Data Set, the bank additional-full.csv. As the data is skewed I am also interested in recall. I am getting accuracy of about 87.95% but my recall is around ...
CommunityBot
- 1
4
votes
1
answer
859
views
Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()
I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
CommunityBot
- 1
2
votes
1
answer
208
views
How do I use wavelet transform for feature extraction correctly?
I'm trying to classify words based on EMG signals using a support vector machine as my model. My dataset includes 15 classes (words) with 230 repetitions and 1000 features each. I already merged all ...
CommunityBot
- 1
3
votes
1
answer
573
views
Gaussian Naive Bayes (GaussianNB) classifier not working with large number of features
I'm trying to make a partial fitting with GuassianNB here's small snippet of my code
...
CommunityBot
- 1
0
votes
1
answer
139
views
Scikit-learn and TensorFlow with very different MLP models
I'm using Multilayer Perceptron ANNs at the very beginning of my project (it's a binary classification problem). Because it's simpler, I started with Scikit-learn. I got a magic result, with my model ...
- 3
0
votes
0
answers
12
views
How to deal with missing values in the output for XGBoost Regressor
So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value.
This is because ...
- 1
2
votes
1
answer
182
views
Not able to encode multiple categorical columns at once
I have written the following code for encoding categorical features of the dataframe( named 't') -
...
CommunityBot
- 1
1
vote
1
answer
24
views
How do I best approach a multiple-target binary classification in Tensorflow/Keras?
I currently have eight features which are either categorical or continuous variables. My targets are many (~1000) binary variables. So far I have attempted skmultilearn and sklearn.multioutput. I ...
- 23.8k
1
vote
1
answer
84
views
Get how similarity between the training data and the income data?
I'am trying to use Clustering and Classification methods as SVM using scikitlearn. I'm also studying some outliers/novelty detections
I want something like a semi-supervised model. I want to predict ...
CommunityBot
- 1
6
votes
2
answers
197
views
Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?
I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning.
Should I include the prediction matrix (ie. df containing columns of prediction results ...
CommunityBot
- 1
1
vote
1
answer
30
views
Scikit-Learn classifiers have impressively bad accuracy on test set for binary text classification problem
I'm trying to fit a GaussianNB and a LinearSVC to binary labeled text using scikit-learn.
To do that, I'm using a TfidfVectorizer to transform my sentences into a matrix of features.
This is ...
- 11
2
votes
1
answer
510
views
Dummy Variable trap in Linear Regression
The dummy variable trap is a common problem with linear regression when dealing with categorical variables, since one hot encoding introduces redundancy, so if we have m categories in our categorical ...
CommunityBot
- 1
10
votes
3
answers
28k
views
How to split train/test datasets having equal classes proportion
I would like to know how I can split in an equal number the following
Target
0 1586
1 318
in order to have the same proportion of 0 and 1 classes in a ...
- 6,082
0
votes
1
answer
146
views
Why there is no alpha parameter for GaussianNB()?
Why there is no alpha argument ( smoothing parameter in Laplace smoothing) for GaussianNB() in sklearn library? ? Although BernoulliNB() and MultinomialNB() have an alpha parameter but GaussianNB() ...
CommunityBot
- 1
0
votes
2
answers
251
views
Model Predictive Power and better prediction of 1 or 0 in scikit-learn?
I have two questions about the Logistic Regression model in scikit-learn:
Which statistic can show me model predictive power?
Which statistic can show me whether my model better predicts event 1 or ...
CommunityBot
- 1
3
votes
2
answers
3k
views
Constraining linear regressor parameters in scikit-learn?
I'm using sklearn.linear_model.Ridge to use ridge regression to extract the coefficients of a polynomial.
However, some of the coefficients have physical ...
0
votes
1
answer
104
views
Multiclass classification dataset with many features producing bad accuracy of predictions
I have been trying to fix this for 2 months now with no luck. I am doing some medical research for my study. I have a dataset that has patients diagnosis based on medical reports (Features.csv) and ...
CommunityBot
- 1
1
vote
1
answer
68
views
Optimize F-Score only for certain classes, disregard other classes
I have a labeled dataset of product reviews where the label is a rating between 1 and 5 and the review is just text.
I use a simple naive Bayes classifier (sklearn) to try to predict a rating given a ...
CommunityBot
- 1
0
votes
0
answers
14
views
NLP approach for classifying webscraped data
I have a challenge in a project of mine where I will be provided with a list of scraped datas from a website. Along with the data i will also be provided some parameters like the tag of scraped ...
- 1
0
votes
1
answer
3k
views
Why can't we feed datetime to Linear Regression and how does toordinal() different from any other integer datatype?
I'm working with dates for the first time. First I knew I had to convert it to timestamps which gave me the values in "datetime64" values. But then I came to know that Linear Regression from ...
CommunityBot
- 1
1
vote
2
answers
449
views
Metrics values are equal while training and testing a model
I'm working on a neural network model with python using Keras with TensorFlow backend. Dataset contains two sequences with a result which can be 1 or 0 and positives to negatives ratio in dataset is 1 ...
CommunityBot
- 1
0
votes
1
answer
8k
views
How to fix "Expected sequence or array-like"
I am trying to get the accuracy of the model and I am getting this error
TypeError: Expected sequence or array-like, got
Here's my code sample.
...
CommunityBot
- 1
4
votes
1
answer
2k
views
Which algorithm is used in sklearn SGDClassifier when modified huber loss is used?
The documentation says:
The loss function to be used. Defaults to ‘hinge’, which gives a
linear SVM. The ‘log’ loss gives logistic regression, a probabilistic
classifier. ‘modified_huber’ is ...
CommunityBot
- 1
1
vote
2
answers
79
views
Does it make sense to train data in scikit-learn and copy+paste parameters into Rust's linfa?
I have a situation where my data can only be read from in a hosted Python environment, due to data security reasons. However, I am constrained to run ML models in a Rust environment due to work-...
- 1,000
2
votes
2
answers
4k
views
Feature scaling for MLP neural network sklearn
I am working with a dataset where the features have multiple scales. Before running scikit-learns's MLP neural network I was reading around and found a variety of different opinions for feature ...
CommunityBot
- 1
0
votes
1
answer
644
views
Precision, recall and accuracy metrics significantly different between training/validation and actual predictions
I have two sequential models built with Keras that train on data from a CSV file.
This is how they are built
...
CommunityBot
- 1
2
votes
1
answer
1k
views
How to impute using simple imputer (custom function)
I am imputing my data using simple imputer from sklearn. i want to test many different ways of applying transformations to the data. i.e for logisitcic regression i would like to
remove nans and ...
CommunityBot
- 1
0
votes
0
answers
8
views
LGBM handle categorical variable with categories which have quantities less than that of min_data_in_leaf
I am building a LGBM model where categorical features have been encoded using ordinal encoding. The categories get values from number from 1 to a max number that are all consecutive. How does LGBM ...
1
vote
0
answers
16
views
1
vote
1
answer
136
views
What is the difference between the CCA weights and rotations?
I have been looking at the scikit learn Canonical Correlation Analysis (CCA) algorithm, and I have come across the terms "weights" and "rotations" as parameters of the CCA model. ...
CommunityBot
- 1
0
votes
2
answers
749
views
Random Forest - Explanation Parameter
I got some question about the "standard" parameter from a random forest. Following I write my understanding about these parameters. I would be glad if I could confirm my understanding or correct it. :)...
CommunityBot
- 1
2
votes
1
answer
124
views
Unbalanced multi-class : distribution might change as more data come in
I am currently working on a problem of multi-class classification on testing logs data.
Basically, I have the context data from tests' execution saved, and want to automate the analysis of the ...
CommunityBot
- 1
0
votes
1
answer
76
views
Correlation with target variable for regression problem
Given the following dataframe
age job salary
0 1 Doctor 100
1 2 Engineer 200
2 3 Lawyer 300
...
with ...
CommunityBot
- 1
1
vote
1
answer
235
views
How to use a Multinomial Naive Bayes Classifier on different sets of data?
I am working on a sentiment analysis project involving tweets. I used a Kaggle dataset to train my model for sentiment analysis and want to use that trained model to predict the sentiment on an ...
CommunityBot
- 1
0
votes
1
answer
452
views
Reading a model stored as binary
I'm using a s3 bucket to store a model I trained in python. Since I'm using an s3 bucket I convert the file to binary first and then store it on the bucket.
...
CommunityBot
- 1