Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

378 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
37 views

weightage attributes

If I have a dataset with binary classification and has many attributes with value of (0 or 1) means the occurrence of attribute is represented by 1 and absence is represented by 0, can I add weight ...
4
votes
2answers
849 views

Combining Machine Learning classifier with NLTK Vader for Sentiment Analysis

As a part of my university project, I am researching/developing a sentiment analysis model wherein I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning ...
4
votes
1answer
405 views

How to implement patternet in python as it is in matlab?

NOTE: This question was first posted in cross-validated website but I was instructed to move it off that website as it was not a good fit. I am new in implementation of machine learning, neural ...
4
votes
0answers
10k views

Tuning Gradient Boosted Classifier's hyperparametrs and balancing it

I am not sure if it is a correct stack. Maybe I should have put my question into crossvalidated. Nevertheless, I perform following steps to tune the hyperparameters for a gradient boosting model: ...
3
votes
0answers
31 views

Data prediction using scikit-learn and a list

I have a group of lists detailing temperatures over differing amounts of time. My goal is to use machine learning to identify periods in which a machine is turned on and off, where turning on the ...
3
votes
1answer
373 views

K Nearest Neighbour with different distance matrix to each datapoint

I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
3
votes
1answer
126 views

ML regression poor performance

I am experimenting with 3 years time series electrical demand data (kW) for a building and attempting to create regression supervised ML models from sci kit learn regressor algorithms but I have very ...
3
votes
1answer
333 views

How many coefficients does the Logistic regression model has as a function of the number of features?

I have built a logistic regression model using Python anaconda and was surprised to see that the number of model coefficients turned out to be proportional to the training sample size i.e. My ...
3
votes
0answers
264 views

How to handle preprocessing (StandardScaler, LabelEncoder) when using data generator to train?

So, I have a dataset that is too big to load into memory all at once. Therefore I want to use a generator to load batches of data to train on. In this scenario, how do I go about performing scaling ...
3
votes
0answers
127 views

PCA and FastICA in scikit-learn giving near identical results

So after importing my data, transforming it, and splitting into training and test sets I tried running this script for PCA: ...
3
votes
1answer
289 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
3
votes
1answer
208 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
3
votes
0answers
2k views

Isolation Forest Feature Importance

As of scikit-learn version 0.19.1, there is no implementation for calculating feature importance in an Isolation Forest. I'm also having trouble finding any online resources proposing ways to get at ...
3
votes
0answers
69 views

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results ...
3
votes
0answers
597 views

Scikit learn: which regressors natively support multi-target regression?

The docs on sklearn.multioutput.MultiOutputRegressor state that it implements a strategy for extending regressors that do not natively support multi-target regression. I'm interested to know: which ...
3
votes
0answers
373 views

Can I do incremental learning with the sklearn implementation of Linear Discriminant Analysis

I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn ...
3
votes
0answers
893 views

Fuzzy Rules with more than two variable in python

I am trying to build a fuzzy inference system in python using skfuzzy library. I have 4 variables depending on which output class is decided. ...
2
votes
0answers
15 views

Logic Check: Building a SKLearn Pipeline

I am new to the concept of building a pipeline in SKLearn and would appreciate some sense-checking to ensure that I am not leaking info from my training sets into my test set. Background: I have a ...
2
votes
0answers
15 views

GuassinaNB parital fit not working properly

I'm trying to make a partial fitting with GuassianNB here's small snippet of my code ...
2
votes
0answers
18 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best hyperparameters ...
2
votes
0answers
48 views

Multi-output regression python

I am new to machine learning and cannot figure a way to solve this problem: I have X which is always one row/record while the y can be a row or more. Here is a simple sample from my data-set, 'i ...
2
votes
2answers
38 views

Clustering list of list of integers

I have ~100 sets of samples with integer IDs. For example, 3 of them could be: a = [0, 1, 3, 4, 6...] b = [1, 5, 9, 102...] c = [1, 7, 10, 42...] I am looking to ...
2
votes
0answers
76 views

Why does classifier chain ask for at least 2 classes, when I have it

I'm using Classifier Chain with logistic regression and when i try to use fit, i get This solver needs samples of at least 2 classes in the data, but the data contains only one class: 1 but I'm ...
2
votes
1answer
70 views

Is there any optimal way on feature selection for more than one classification algorithms?

I have a wine dataset with 13 features that indicates 3 different wine classes (target), and k-NN, SVM with linear kernel and SVM with rbf kernel algorithms to be tried with this dataset. My goal is ...
2
votes
0answers
27 views

What is the scikit learn Non-negative Matrix Factorisation Coordinate Descent algorithm?

What is the scikit-learn Coordinate Descent (CD) algorithm for Non-negative Matrix Factorization (NMF)? The sklearn implementation of NMF has two different solvers, Coordinate Descent and ...
2
votes
0answers
96 views

Linear Regression on data with bimodal outcome

I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
2
votes
0answers
15 views

Temporal outlier Analysis on sensor data

I am working to find anomaly/outliers in sensor data using unsupervised machine learning (without training dataset). I have around 20000 samples taken per minute of various sensors. I just need to ...
2
votes
0answers
26 views

different outcome of feature importance and coefficient from same data

I built a regression model to predict profit based on client, sales person, product category, client industry and client region. After trying several models with tuning hyperparameters, I found that ...
2
votes
0answers
62 views

Comparing feature importance in LightGBM + Scikit

I have a model trained using LightGBM (LGBMRegressor), in Python, with scikit-learn. On a weekly basis the model in re-trained, and an updated set of chosen features and associated ...
2
votes
1answer
228 views

incremental learning vs transfer learning

Can anyone explain me how incremental learning differs from transfer learning with example? Also does Transfer learning limited to neural networks?
2
votes
0answers
79 views

Why are the regions/decision boundaries overlapping with multi-class classification using SVM in sci-kit?

I am using the SVM in scikit-learn library for doing multiclass classification. I am wondering why these regions (decision boundaries) are overlapping (as seen in the picture below)? Could someone ...
2
votes
1answer
951 views

memory error in matrix cosine_similarity

I have (20905040, 7) of a dataset to recommend 10 different product to the user it could be larger than that but anyway I got memory error when processing the ...
2
votes
1answer
32 views

How can I count the number of occurrences of a category in dataset as part of an Sklearn Pipeline

Let us say we have a dataset with a feature such as Surname. arr['Surname'] = ['Smith', 'Jones', 'Johnson', 'Smith'] And I want to encode this categorical info ...
2
votes
1answer
45 views

Item-based recommender using K-NN

I'm trying to build an item-based recommender using k-nn. I have a list of items, all of which have some properties (features) in common. ...
2
votes
0answers
11 views

Confusion regarding prediction results of SVM and ANN on feature vectors

I am making a custom image classifier using Transfer Learning on Inception V3. I have 3 classes of images with ~6K images each. The input dimension of the network is 500X500 and the output of the ...
2
votes
1answer
25 views

Sensorfusion: Generate virtual sensor based on analysis of sensorsdata

I have a steam engine which is equipped with the following sensors: temperature sensor in the boiler room temperature sensor in the heating room pressure sensor in the boiler room rotations-per-...
2
votes
1answer
1k views

K-fold cross validation of scikit-learn with confusion matrix of Keras

I intend to display confusion matrix using Keras while K-fold of scikit-learn. My code using Keras is: ...
2
votes
0answers
91 views

How to improve a model with a high cross validation score yet with low accuracy on unseen data?

I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows: Record raw data from the subjects For each subject, compute the feature from the raw data ...
2
votes
0answers
78 views

Non-linear Support Vector Regression issue - Sklearn Python 3.6

I am fairly new to Sklearn and machine learning and have encountered an issue when using SVR with an RBF kernel. Below is my predicted data compared directly with my real data: I do not know what I ...
2
votes
0answers
20 views

Varying strength of prior for MCMC hierarchical linear model

I am training an MCMC model in using Pymc3. My aim is to build a series of linear regression models which will predict the time to unload a truck, based on the number of crates to unload. I have ...
2
votes
1answer
442 views

How to deal with missing data for Bernoulli Naive Bayes?

I am dealing with a dataset of categorical data that looks like this: ...
2
votes
0answers
59 views

Unbalanced multi-class : distribution might change as more data come in

I am currently working on a problem of multi-class classification on testing logs data. Basically, I have the context data from tests' execution saved, and want to automate the analysis of the ...
2
votes
2answers
561 views

How can I find anomalies in each row of data?

I have some reported data I want to spot anomalies on. The columns are a facility name then monthly reports of that given facility. ...
2
votes
1answer
765 views

Low silhouette coefficient

I am doing a kmeans clustering on a dataset of selling values of articles. Each article has 52 selling values (one per week). I am trying to automatically calculate the optimum amount of clusters ...
2
votes
0answers
58 views

logistic regression

I am writing a scientific paper that - among other things - deals with logistic regression in the context of machine learning. I read this article where the author states that, given a set of instance-...
2
votes
2answers
5k views

Creating Balanced Dataset Using Scikits

I have a classic User-Item dataset where each row (i.e., (user, item)) indicates the action of a user clicking/selecting an item....
2
votes
0answers
36 views

Term for Methods of Representing Repeated Text in Classifier

A colleague told me that there are terms for two different methods of representing repeated text in the training set for a classifier, but he could not recall them. What are the terms for the options ...
2
votes
0answers
162 views

How to create a global model with personalized features for multi-label classification problem

I'm trying to predict additional recipients of a message given the content of the message (like subject and body) and the current recipients of the message. for ex: I have 4 users in the system U1, ...
2
votes
0answers
792 views

Large sparse dataset in Catboost

I have a large sparse data matrix (bag of words, over large number of entries). I can easily treat it as a sparse matrix in sklearn models such as ...
2
votes
0answers
997 views

How to tune weights in Voting Classifier (Sklearn)

I am trying to do the following: ...

1 2 3 4 5 8