Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [scikit-learn]

Scikit-learn is a Python module comprising of simple and efficient tool for machine learning, data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. It is distributed under the 3-Clause BSD license.

0
votes
1answer
10 views

A bit confused regarding clustering of users in a dataset

I have a dataset of book reviews: ...
0
votes
1answer
10 views

sklearn stratifying dataset with more than one label

I have come across sklearn.model_selection.train_test_split as a method to split up the train and testing dataset. Furthermore they have a ...
1
vote
1answer
31 views

Random forest vs. XGBoost vs. MLP Regressor for estimating claims costs

Context I'm building a (toy) machine learning model estimate the cost of an insurance claim (injury related). Aim is to teach myself machine learning by doing. I have settled on three algorithms to ...
2
votes
2answers
185 views

Is it OK to use the testing sample to compare algorithms?

I'm working on a little project where my dataset have 6k lines and around 300 features, with a simple binary outcome. Since I'm still learning ML, I want to try all the algorithms I can manage to ...
2
votes
1answer
25 views

Problems in building model predicting probablity of student's admission

I'm trying to build a model predicting the probability of a student's admission in Russian educational system. There are exams for every school subject. The student can take several of them. Based on ...
3
votes
1answer
25 views

K Nearest Neighbour with different distance matrix to each datapoint

I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
0
votes
0answers
12 views

sklearn & Meanshift for NLP only returns 1 cluster

I am using sklearn.clustering to work with some text data and the MeanShift algorithm. I have: Done all standard NLP data prep like lemmatizing, removing stop ...
2
votes
1answer
27 views

algorithm to predict numeric values with sklearn

I'm new to ML and trying to learn it. I scraped information (using python) from a website where people try to sell their cars. I put data in a pandas data frame. Now I'm wondering how should I predict ...
1
vote
1answer
23 views

Why is my MLP with 2 features is doing worse than MLP with 1 feature where the one feature is a combination of feature1*feature2?

I have programmed a MLP for a dataset (~500 rows) containing the length (L) and width (W) of an organism and the output of biomass (the organisms weight in pounds, B). ...
0
votes
0answers
23 views

How to test multiple algorithms at once?

I was wandering if there is a method with Python and/or Sklearn to test multiple algorithms at once instead of run them one by one and see the accuracy. I have been looking on the web and I've found ...
0
votes
1answer
12 views

correct setting of eval_set in multiclass classification xgboost python , error is “ Check failed: preds.size() == info.labels_.size()”

i have a multiclass classification problem with 3 classes [-1,0,1] . i'd like to use eval_set in xgboost. but it fails with error: ...
0
votes
0answers
28 views

New images always predict one label

I have trained a SVM for image classification using RGB histogram as features and a couple of other ones. These are my feature and label sizes: ...
1
vote
1answer
36 views

What does sklearn's pairwise_distances with metric='correlation' do?

I've put different values into this function and observed the output. But I can't find a predictable pattern in what is being outputed. Then I tried digging through the function itself, but its ...
1
vote
1answer
43 views

What happens to a machine learning technique (specifically Decision Tress and Logistic Regression) if the validation dataset has a new category?

Let's suppose I have a dataset which has a categorical variable and the problem I am solving is a classification one. This categorical variable var has ['A','B','C'] as the possible set of data. ...
1
vote
0answers
35 views

RandomForestRegressor intermittantly returning a single prediction

BACKGROUND I have a RandomForestRegressor from scikit-learn which, for each example row, takes in four float features and ...
0
votes
0answers
14 views

Is the mean shift algorithm adapted to my problem?

I'm currently building a model that can detect abnormalities in a timeserie. First, we predict the next steps and then we compare the prediction with what we measure in real time. We want to see if ...
1
vote
2answers
29 views

Labeling classes conditionally

I am working with a time series predicting whether web traffic will increase or decrease each day compared to the previous day for a given user. Initially I used binary classes: labeled 1 for next ...
0
votes
1answer
43 views

predict_proba returns different results on Python 2 & 3

I had some old code used to train a Random Forest Classifier (sklearn 0.17.1), for classification on two classes (spam/ham). I ran this in a docker container and sent it some data. Sklearns ...
0
votes
2answers
53 views

Standardise AND Normalise

I am new to machine learning. Does it make sense that my model works when I do both Standardise and Normalise? What does this say about my data? Or do I do have to select one or the other? My goal ...
3
votes
1answer
21 views

Is it possible for a neural net to score as high as a different form of supervised learning?

I've been working with the Adult Census Income dataset from UCI http://archive.ics.uci.edu/ml/datasets/adult I've created two different models, one using a gradient boosted classifier with sklearn, ...
0
votes
0answers
24 views

How to pass inputs (interactively) to a model?

Let me give you a high-level design (blueprint) of my model. Input data:: ...
0
votes
1answer
38 views

Advice on transitioning from Andrew Ng's Stanford Coursera Machine Learning (in MATLAB) into Python?

I'm currently finishing up Andrew Ng's Coursera course, taught in MATLAB/Octave, but I'm looking to code in Python. The course is an introduction into how some algorithms work from scratch. These ...
0
votes
1answer
24 views

Checkpoints in Sklearn

Is there a way to save the current state of your experiment so that you can pick up from where you left off in Sklearn similar like checkpoints in Keras ?
0
votes
0answers
30 views

XGBOOST (sklearn interface) REGRESSION error

I am trying to run a GRIDSEARCHCV (sklearn) on XGBRegressor. Documentation on the parameter says that if regression, then objective = reg:squarederror.(see https://...
0
votes
0answers
18 views

How can we add preprocessing steps, in the keras sequential model itself?

Is there a way to add a layer which includes my preprocessing steps in this sequential model.For example ...
0
votes
1answer
92 views

Classify sensor data (multivariate time series) with Python's scikit-learn decision tree

i'm trying to apply scikit learns decision tree on the following dataset with the goal of classifying the data: sensordata: multiple .csv files every .csv file has multiple sensors (see here) each ....
1
vote
1answer
24 views
1
vote
1answer
32 views

Residual plot is tight to zero, but low R2 score

Hi I am a beginner at data science, and currently trying to use Gradient Boost Regressor to predict car price based on several attribute such as machine capacity, car brand, car type etc. I am ...
0
votes
1answer
29 views

Machine learning with sklearn vs. scipy stats

I've created 50 random x and y points (with slope of y = 2x-1). First, I used Linear Regression from sklearn to fit the model onto my dataset where I got a slope of ...
0
votes
1answer
25 views

Why is the reported loss different from the mean squared error calculated on the train data?

Why the loss in this code is not equal to the mean squared error in the training data? It should be equal because I set alpha =0 , therefore there is no regularization. ...
0
votes
1answer
22 views

Error: ValueError('%r cannot be used to seed a numpy.random.RandomState')

I am getting this error message while trying to fit a model for the isolationForest algorithm. ...
0
votes
2answers
34 views

Random Forests Feature Selection on Time Series Data

I have a dataset with N amount of features, each one with 500 instances in time. Let's say that I have for example, the features x, y, v_x, v_y, a_x, a_y, j_x, j_y....
0
votes
0answers
9 views

SKlearn Univariate feature selection if the features are continuous and output is categorical

I have a dataset with 200 numerical features, but the target is binary (0 or 1). If I use uni-variate feature selection, What is the right scoring parameter (f_regression or f_classif /chi square)? ...
0
votes
1answer
60 views

Feature selection - SelectKBest sklearn

I would like to ask how to set paramater k in function SelectKBest for feature selection. I have now around 2300 features, so I think that default value 10 is not enough. Is there any approach, how ...
2
votes
1answer
28 views

Python - Create many dummy variables from one text variable?

I'm trying to create dummy variables for a variable that has text data in rows. Data in 1st row is: ...
0
votes
0answers
15 views

Binary Classifier for photo detection

Two training sets are involved, one complete, one with missing feature data as well. The data consists of CNNs and GIST features. For the normalising, I have MinMax Scaler feature. I have cleaned up ...
1
vote
1answer
22 views

Can I accurately call sklearn.model_selection.train_test_split multiple times when data doesn't fit into memory?

Consider a very large data set that doesn't fit into memory. Would I be able to get (nearly) the same behavior from multiple calls to train_test_split when calling train_test_split by passing batches ...
1
vote
1answer
34 views

Multilabel classifcation in sklearn with soft (fuzzy) labels

I have a model which is trained in sklearn on a 5-way classification problem, which performs relatively well (there are kNN and SVM versions, and both reproduce a test set with high accuracy). When ...
4
votes
1answer
25 views

How to use a one-hot encoded nominal feature in a classifier in Scikit Learn?

Im working on a genre classification problem on a songs dataset. Since genre is a nominal feature, I used sklearn's LabelBinarizer to get the one-hot encoding for this feature for every row in the ...
2
votes
2answers
50 views

sklearn and pandas in AWS Lambda

I made a front end where I would like to make REST calls to an AWS Lambda interfaced with AWS API Gateway. I dumped my model as a pickle file (and so my encoders) which I initially trained locally. ...
2
votes
1answer
44 views

Not able to interpret decision tree when using class_weights

I'm working with an imbalanced dataset. I'm using a decision tree (scikit-learn) to build a model. For explaining my problem I've taken iris dataset. When I'm setting class_weight=None, I understood ...
1
vote
0answers
12 views

Out of sample extension for Isomap in Sklearn

If I'm fitting the isomap class with a certain dataset, then I transform with a different one, does that mean that Sklearn is doing out-of-sample extension ? I.e. ...
1
vote
2answers
55 views

Low accuracy in multi-class classification despite all data being generated from rules

I have a well defined data where i have cleaned up my data to final form which has 20 features mapping to a number between 1 to 100. Upto 5 features are enabled(value set to 1) for each row. The data ...
0
votes
1answer
15 views

When to question output of model

I'm unsure of how to ask a question without making it seem like a code review question. At what point does one question whether they've actually implemented the algorithm and-or model correctly? ...
1
vote
1answer
43 views

neural network to find a very simple linear model (scikit-learn)

I'm trying to test different machine learning algorithm to try to find correlation between various data on MRI scans. Since I'm dealing with medical data, I don't have access to many events, but still ...
0
votes
1answer
30 views

Collinearity and Multicollinearity in the features?

What are some advanced or basic methods most used by data scientists/ML Engineers to detect collinearity (or) multicollinearity between features?
3
votes
1answer
30 views

What are the cases in which Isomap fails to do a good job?

As above, what is a possible scenario/ dataset/ case in which Isomap fails to do a decent dimensionality reduction?
1
vote
0answers
31 views

Linear regression load model doesn't predict as expected

I have trained a linear regression model, with sklearn, for a 5 star rating and it's good enough. I have used Doc2vec to create my vectors, and saved that model. Then I save the linear regression ...
0
votes
2answers
23 views

How do I correctly build model on given data to predict target parameter?

I have some dataset which contains different paramteres and data.head() looks like this Applied some preprocessing and performed Feature ranking - ...