Questions tagged [scikit-learn]

scikit-learn is a popular machine learning package for Python that has simple and efficient tools for predictive data analysis. Topics include classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.

Filter by
Sorted by
Tagged with
0 votes
0 answers
32 views

My machine learning model cannot classify data that it has never seen before

I am creating a spam identifier to detect wheter an email is malicous. The issue I have is my model using the RandomForest Classifier showed it was 99% accurate. csv: https://www.kaggle.com/datasets/...
John Adams's user avatar
0 votes
0 answers
22 views

Algorithms from R-statistics package Caret R Package- LVQ algorithm, is there similar in Python

In the R-statistics package : Caret R Package, they have the LVQ algorithm that is used for the purpose of "Feature Selection". I have used this to do some data science in R-stats over 6 ...
Palu's user avatar
  • 103
0 votes
1 answer
33 views

Python SK-Learn KNN Imputer ( "ValueError: could not convert string to float: )

I have data with missing values. All columns are integer, except for a column that has missing values. These missing values, were set with a "?" which was converted to NaN using the Numpy ...
Palu's user avatar
  • 103
1 vote
1 answer
19 views

Number of stop words variation in libaries sklearn and nltk

Is there a reason why there is a big variation in the number of stop words? I assumed that there would be a general agreement from English experts how many stop words there could be. And even with ...
heretoinfinity's user avatar
0 votes
2 answers
340 views

Solve a non-linear system, in Python, with the GAUSS-NEWTON algorithm? (Jacobian matrix J, etc.)

I would like to solve a non-linear system (which contains the goals of a football team in previous matches) using the Gauss-Netwon algorithm, in order to find the parameter (of frequency) to use as ...
Frimand's user avatar
  • 59
0 votes
1 answer
11 views

Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?

I have been working with Python for machine learning and have a fair amount of code written in Python using libraries such as scikit-learn, pandas, and numpy. Recently, I’ve been faced with larger ...
Mohith7548's user avatar
1 vote
1 answer
21 views

Out-of-Range Target Variable in Sequence-based Machine Learning Model

I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the ...
Bloggy's user avatar
  • 11
0 votes
0 answers
13 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
1 vote
1 answer
26 views

How do I best approach a multiple-target binary classification in Tensorflow/Keras?

I currently have eight features which are either categorical or continuous variables. My targets are many (~1000) binary variables. So far I have attempted skmultilearn and sklearn.multioutput. I ...
FoolsGold1997's user avatar
1 vote
1 answer
32 views

Scikit-Learn classifiers have impressively bad accuracy on test set for binary text classification problem

I'm trying to fit a GaussianNB and a LinearSVC to binary labeled text using scikit-learn. To do that, I'm using a TfidfVectorizer to transform my sentences into a matrix of features. This is ...
Towdo's user avatar
  • 11
0 votes
0 answers
14 views

NLP approach for classifying webscraped data

I have a challenge in a project of mine where I will be provided with a list of scraped datas from a website. Along with the data i will also be provided some parameters like the tag of scraped ...
LEO_007's user avatar
1 vote
2 answers
81 views

Does it make sense to tune a model in scikit-learn and copy/paste the parameters into Rust's linfa?

I have a situation where my data can only be read from in a hosted Python environment, due to data security reasons. However, I am constrained to run ML models in a Rust environment due to work-...
wtwtwt's user avatar
  • 111
0 votes
0 answers
9 views

LGBM handle categorical variable with categories which have quantities less than that of min_data_in_leaf

I am building a LGBM model where categorical features have been encoded using ordinal encoding. The categories get values from number from 1 to a max number that are all consecutive. How does LGBM ...
bunti papu's user avatar
1 vote
0 answers
16 views

sklearn - OneHotEncoding and SelectPercintile

in sklearn example there is a code ...
Maciej778's user avatar
1 vote
1 answer
29 views

Scale dataset while preserving relative distributions between columns

I have a large dataset with 460 columns. The columns have names such as 'AppOpen_1day', 'AppOpen_2day', ...... 'AppOpen_15day', 'Dig_Pos_1day', 'Dig_Pos_2day', ...... 'DigPos_15day' etc. Each column ...
SacredMechanic's user avatar
4 votes
1 answer
735 views

Decision Tree only splits to the left

I can’t really understand, why my decision tree only splits to the left. I originally have 2 categorical features (further named feature 0 and 1), which I concat to one feature since feature 1 is ...
Taitex's user avatar
  • 41
0 votes
1 answer
37 views

How to explain the new features after a PCA?

Let's say I made a PCA in which I reduced from 10 dimensions to 3. And it clusters the classes correctly, but how do I explain which dimensions are better to predict? It is obvious that the 3 ...
MAD MAGGOT's user avatar
1 vote
2 answers
183 views

Why do we need hyperparameter tuning in Scikit learn? Doesn't sk learn models by default give best model?

When I have the option to build a classifier like this directly clf = RandomForestClassifier() why do we perform tuning by restricting the parameters like this <...
Hola's user avatar
  • 13
0 votes
0 answers
22 views

Logistic regression with E-net regularization produces different set of weights with each run

I am currently trying to make a model to classify brain tumor patients by incidence of epilepsy using a combination of variables extracted from clinical records, and radiomics features from segmented ...
reuben george's user avatar
0 votes
1 answer
44 views

How to assign sample weight for regression problem

I'm trying to model a forecasting problem where I'm trying to forecast for the following month. I am using LightGBM Regressor class for it and it giving me a decent ...
Krishnang K Dalal's user avatar
0 votes
0 answers
31 views

The optimal way to stratify a numerical target variable into a categorical one for a machine learning algorithm

I have tabular data, the predictive variables are numerical and categorical and the target variable is a numerical one. Using the proper techniques I can make predictive models with R^2=0.95. Now let'...
ADayWithoutRain's user avatar
0 votes
0 answers
46 views

K nearest neighbor with varying feature length

I'm trying to build a tool which predicts the elemental composition of some light source using its emission spectrum with the k nearest neighbor algorithm (I'm using the KNeighborsClassifier from ...
oodani's user avatar
  • 1
0 votes
0 answers
20 views

How are the successive sets of training samples that are allocated for each iteration of HalvingGridSearchCV determined?

The scikit-learn classes HalvingGridSearchCV and HalvingRandomSearchCV implement a hyperparameter tuning method known as successive halving. It is an iterative selection process in which all the ...
Evan Aad's user avatar
  • 101
1 vote
0 answers
23 views

How to know which rules were applied to predict one sample in trained decision tree model?

I have trained Random Forest Regressor from sklearn. I am able to return text representation from each Decision Tree rule using tree.export_text (sklearn documentation here). But it shows rules for ...
Paulina's user avatar
  • 15
0 votes
1 answer
34 views

Different accuracy scores with sklearn roc_auc_score on same model using sklearn.metrics

Why do these below lines give different outputs while the input is the same? I need to report these results in paper, but I am unsure which is better and why. ...
Adnan Ali's user avatar
  • 101
0 votes
0 answers
22 views

How to Fix Dimension Issues of features and classes from a Multilabel Classification dataset in getting the Out-of-Bag Error of a Random Forest?

I have created a multilabel classification dataset using make_multilabel_classification from scikit learn: ...
Ralph Henry's user avatar
0 votes
0 answers
12 views

Hot-encoding warning when using gridsearch

I ran an experiment with the classical holdout method to predict price and hot-encoded categorical data. However, when optimising, I got the warning below even though that I ignored the unknown ...
Aze 's user avatar
  • 1
0 votes
0 answers
12 views

Why do my tf-idf values not appear consistent?

I have a series of tweets that I've converted to tokens. Among them are the following: geraldkutney happen realize happen conveniently rename catch yet emergency post fact come government ...
Daniel V's user avatar
  • 101
0 votes
0 answers
42 views

Low silhouette score yet better clustering result

I applied k-means and k-medoids clustering techniques on iris dataset, in particular I clustered with respect to sepal length and sepal width features across 4 classes. With k=3 using k-means I ...
Harshavardhana Srinivasan's user avatar
0 votes
0 answers
33 views

XGBoost Classifier + Isotonic Regression leading to worse probability accuracy

I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
Ted's user avatar
  • 1
0 votes
0 answers
51 views

Using "precomputed" distance matrices as input to scikit-learn clustering metrics

Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
Chris Coffee's user avatar
1 vote
1 answer
13 views

Classifier learn from model or data

I'm new in data science. I read context . It saids "The clf (for classifier) estimator instance is first fitted to the model; that is, it must learn from the model. ...." and example shows: <...
Mike Liu's user avatar
0 votes
2 answers
19 views

How to encode Income Type Ordinal Data into numbers?

I am doing a mini project on Credit card Approval Prediction. The Dataset I have taken is from Kaggle: https://www.kaggle.com/datasets/rikdifos/credit-card-approval-prediction Problem: I want to ...
Prajwal Dhage's user avatar
0 votes
0 answers
13 views

scikit GMM fails randomly with ill-defined linear covariances

I am trying to fit a GMM model to my data. The dataset has 37 features (some int and others float). When the dataset has small number of rows (<200), GMM fits OK. When I try a larger dataset (500 ...
RedBaron's user avatar
  • 101
0 votes
0 answers
11 views

clustering droplet data along trajectories

I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
Jonas Hilti's user avatar
-1 votes
1 answer
101 views

ARMA model using different train and test/validation datasets

In sklearn I am used to having distinct train and test datasets. In other words, I train a model's parameters on the features from a training set, and then apply ...
user3128's user avatar
1 vote
0 answers
14 views

Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation

Question: I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
sara sara's user avatar
0 votes
0 answers
15 views

Macro Averaging vs. Samples Averaging in multilabel classification problems

I am currently working on a multilabel classification problem and I have developed some models to solve it using the SciKit-Learn framework. I wish now to evaluated the models by producing scores for ...
user2566415's user avatar
1 vote
0 answers
113 views

How to split graph data into train and test sets for link prediction problem using node emebbdings and cosine similarity

I would like to predict new links using node embeddings and cosine similarity, but I am unsure how to split the data set into training and testing, and how to evaluate new links. This is my code ...
Ghada Kahlaoui's user avatar
0 votes
0 answers
10 views

Test accuracy plateaus when increasing max_depth -> inf

I've built a Random Forest model that classifies into four categories based on around 10 input features. To test the accuracy, I performed 5-fold stratified cross validation using the ...
okjdlsksjdwi's user avatar
0 votes
1 answer
25 views

Why most keras examples use shape=(None,) instead of shape=(None)?

I'm reading several keras tutorials and found that many examples are written like this: keras.Input(shape=(None,), dtype="int64", name="english")...
skan's user avatar
  • 185
0 votes
0 answers
12 views

Does scikit learns implementation of silhouette score support parallelization and will benefit from multiple CPUs?

I wish to use the silhuette score to get the optimum number of clusters. I know kmeans implementation in scikit learn supports parallelization. But I am unsure whether the same is true for silhouette ...
Ali Raheel's user avatar
-1 votes
2 answers
76 views

Recall and Precision ML models

I use decision trees for a binary classification. To evaluate the model, I use K-fold cross-validation, where k = 10. When I run the model n times, I get a relatively constant accuracy across all ...
Jan Jansen's user avatar
0 votes
0 answers
11 views

Detect Recent Anomalies

I'm trying to see how to approach this problem: I have a dataset of fraud transactions. There are several categorical columns, like country, type, merchant, etc. All of the records are considered ...
F_M's user avatar
  • 131
1 vote
1 answer
31 views

Learning Curves - Entire dataset or only Training dataset?

Can someone please explain if Learning Curves should be plotted using entire dataset (all X and all y) or just using Training dataset (X_train and y_train)?
Arun kumar's user avatar
0 votes
0 answers
14 views

Data processing for Events that repeat a fixed number of times in each group in machine learning

Assume I have a football teams historical match performance data at player level, with attributes like number of goals score by each player, other performance metrices like passes given, crosses done, ...
imhaka's user avatar
  • 1
0 votes
3 answers
169 views

Clusterise a group of answers taking the question as relevant information

I am solving a problem where I group answers to a given question into clusters using k-means algorithm. The steps I follow are: For every answer I get the corresponding vector. Reduce the vector ...
jesantana's user avatar
0 votes
0 answers
20 views

Input shape CNN speech

The shapes of x_train=(514, 256), y_train=(514,) x_test=(254, 256), y_test=(254,) I have 256 features and 770 samples of which ...
Srikanth's user avatar
0 votes
0 answers
29 views

Imputation of multi-peaked dataset

I am a beginner to model fitting, and I have been working on generating a model for a CO2 emissions data set. The distribution of the data points in a number of these columns are very markedly dual-...
Phindolin's user avatar
0 votes
1 answer
33 views

Why does a .fit() method change an input X to a np.array?

Why does a (custom) .fit() method transform a pd.DataFrame X to a numpy.ndarray? ...
Tim's user avatar
  • 5

1
2 3 4 5
47