Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1 vote
1 answer
31 views

Techniques for classification AI with sparse labels

I want to create an AI to classify images with a large set of labels (1000+ labels). However, the labels in the data set are correct but each image is not fully labelled. This means that each image's ...
user avatar
  • 11
1 vote
1 answer
9 views

Sub labelling of an object

First timer in image processing - Pardon my cluelessness. Is there a concept of sub labeling in objection identification? I want to label a person and sub label "eye" of a person and train a ...
user avatar
  • 111
0 votes
1 answer
25 views

Plot distribution of multi classification with features - Python

I have a multi-class classification problem, and continuous features, I want to visualize using python libraries (matplotlib, seaborn, plotly..) the target variable ( 4 classes ) with each given ...
user avatar
  • 21
1 vote
1 answer
33 views

XGboost Classifier predicits different results for same samples depending on test dataset size

I train a simple xgboost classifier model with the following lines. ...
user avatar
  • 41
2 votes
1 answer
26 views

Which Model for predicting flight delays is appropriate except Random Forest and Decision Tree? (Monte Carlo?)

Im studying M.Sc Data Science and in the module "Decision Support Systems" me and my group have to make a presentation. Our Proposal is the following: Background With generally high demand ...
user avatar
  • 21
1 vote
1 answer
30 views

What is the right way of training Regression model having various categories involved?

I am working on one regression problem statement and it involves multiple categories into it. I am not sure how to proceed with it, hence looking for your guidance/suggestions over it. Suppose there ...
user avatar
  • 207
3 votes
2 answers
134 views

Manipulating noise to get some data in right format and apply it to task using PPO

Warning: I understand that my question may seem strange, stupid, and impossible, but let's just think about this interesting problem. I would not ask a question like: how to create an AGI in google ...
user avatar
0 votes
1 answer
24 views

Interpreting a curve val_loss and loss in keras after training a model ; help

I need some help in Interpreting a curve val_loss and loss in keras after training a model These are the learning curves:
user avatar
0 votes
0 answers
11 views

What are the differences between uniform and normal QuantileTransformer Normalizer?

In sklearn.preprocessing package there are QuantileTransformer(output_distribution="uniform") and ...
user avatar
  • 33
0 votes
0 answers
15 views

TypeError : Inputs to a layer should be tensor [migrated]

I am new to deep learning currently trying to learn neural network.However,I encountered this problem while training the neural network. This is the input .I thought by using the tensor Dataset I am ...
user avatar
0 votes
1 answer
20 views

Seaborn Heatmap with month & hour of database entry

I have a dataframe with X rows. For each row, I have the information of the month (value from 1 to 12) and hour (value from 1 to 24) in separate columns I need to create a heatmap with seaborn in ...
user avatar
  • 103
0 votes
0 answers
14 views

Batch size to avoid overfitting

I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
user avatar
0 votes
0 answers
22 views

Is it a right way to score the importance of categorical features against categorical targets using ANOVA?

I know that f_classif method is mostly used for scoring numerical features and categorical targets. But I'm wondering if it's possible to score features coded by integers or binary codes against ...
user avatar
  • 101
0 votes
1 answer
17 views

How to examine effect of variable not used in training a neural network

I am currently using tensorflow to create a neural network that does basic binary classification, and I would like to examine the bias of the model after training. I have a dataset of about 300,000 ...
user avatar
  • 21
3 votes
2 answers
111 views

Binary classifier high overall ROC AUC but low in different bins

I'm trying to analyze the performance of a binary classifier on the test set on different ranges of the predictions. the classifier has a .97 ROC AUC on the test. Then I binarize the test set ...
user avatar
0 votes
0 answers
10 views

how to calculate purity of two clustering methods of unsupervised learning model in python

I want to calculate the purity of clustering of methods which does not have a true label but have a predicted label so that I can compare it to another model. I want to know the algorithm to calculate ...
user avatar
0 votes
0 answers
14 views

Validate different unsupervised learning algorithms and by combination of parameters

I am applying various clustering algorithms on a given data set: CH: Calinski-Harabasz DB: Davies-Bouldin For each algorithm, I first want to obtain the best combination of parameters: ...
user avatar
  • 103
0 votes
0 answers
58 views

Evaluate multiple Isolation Forest estimators during GridSearchCV with custom scorer function

I have a sample of values that don't have a y target value. Actually, the X features (predictors) are all used to fit the Isolation Forest estimator. The goal is to identify which of those X-features ...
user avatar
  • 101
0 votes
0 answers
13 views

When I use imblearn pipeline instead of sklearn pipeline all textual features disappear. Any solution?

This is my code below, I need to use SMOTENC to balance the dataset, which means I have to use the pipeline from the imblearn library. However, it does not recognize the CountVectorizer features ...
user avatar
0 votes
0 answers
16 views

Why do i need to write regressor.predict(x_train)?

Im currently learning data science and i was unable to understand a particular part in linear regression model. The following is my code - ...
user avatar
  • 1
0 votes
1 answer
32 views

precision and recall at k for movielens dataset

I wanted to recreate a very simple collaborative filtering example with the 1M movielens dataset I have from Kaggle (https://www.kaggle.com/datasets/odedgolden/movielens-1m-dataset) and then ...
user avatar
0 votes
0 answers
24 views

Do I need to install apex for fairseq if I have pytorch >= 1.10?

In installation section of fairseq toolkit there are instructions "For faster training install NVIDIA's apex library", but in we have torch.cuda.amp already added into pytorch. So my ...
user avatar
1 vote
0 answers
24 views

How to return selected features with different feature selection models?

I use the below function to detect the effect of those feature selection models on my data, it works perfectly. what I want is to return the name of selected features for each model, is there any ...
user avatar
  • 33
0 votes
0 answers
15 views

How to find location for stretch of values more than fixed number and their average in dataframe python3 [migrated]

I have a data-frame something like this: And I wish to find the the location (start and end) and the average of the stretch of stretch where values in column 2 are more than 0.2. Something like this: ...
user avatar
  • 3
0 votes
0 answers
10 views

Algorythm for the interpolation?

Hi I have now managed to plot various points on a map and interpolate with ordinary kriging. However, my data does not look anything like it should. Do I need to use an algorithm to fill in the ...
user avatar
  • 101
0 votes
1 answer
40 views

Merging two datasets with different features for machine learning prediction

I'm trying to create a model which predicts Real estate prices with xgboost in machine learning, my question is : Can i combine two datasets to do it ? First dataset : 13 features Second dataset : ...
user avatar
0 votes
0 answers
8 views

Turning multiple binary columns into categorical (with less columns) with Python Pandas

I want to turn these categories into values of categorical columns. The values in each category are the current binary columns present in the data frame. We have : A11, A12.. is a detail of A1 so if ...
user avatar
  • 21
1 vote
1 answer
16 views

What kind of model/type is this

Essentially I want to pass a program some variables, all gathered from a user on my site, and have the program give a "score" of how authentic the user is meant to be. I already have a large ...
user avatar
  • 11
0 votes
0 answers
8 views

Why I am getting Infinity infity in LineString?

I am try get linestring so I can measure the distance and time. Here in this linestring I am getting nan distance and time. Also, pleased to hear any of your ...
user avatar
  • 11
1 vote
0 answers
10 views

How to label legit users when trying developing a bot flagging classification model?

I’m working on a project where I try to flag bots from legit users on social media. The data I collected is not labeled but I have labeled about 17% of it (22k users) thought different techniques. ...
user avatar
  • 93
0 votes
0 answers
24 views

How to train LGBMClassifier using optuna [migrated]

I am trying to use lgbm with optuna for a classification task. Here is my model. ...
user avatar
  • 121
1 vote
0 answers
9 views

Input-depending model on TensorFlow

I'm working on a TensorFlow model, and I would like to have different logic in the model depending on the input. My question is if it's possible to have a "dispatch" layer in TensorFlow that ...
user avatar
  • 489
0 votes
0 answers
12 views

Retrive image from from features represented by histograms of oriented gradients

I am using histogram of oriented gradients for image classification using clustering in scikit learn. I am using hog from scikit-image to generate hog from 512x512 grayscale image. Here is an example: ...
user avatar
0 votes
0 answers
19 views

What do you think is the best method (image registration) for the attached image?

I have a set of images for multimodal medical images, and after doing the segmentation, I need to register the green lines with red lines in the attached figure. Which method do you think is best for ...
user avatar
  • 1
0 votes
0 answers
20 views

List of words cluster by topics

I have a list of words, these words correspond to labels in news and they are not duplicated. I would like to get a clustering for this list based on topics. I try with wordnet but I don't know how ...
user avatar
1 vote
1 answer
14 views

How to smooth the Linestring after adding additional point?

Actually I am tracking vessel, where I am already having path, and current position. Now the trouble is, obviously the vessel will not be on same coordinates as provided in the path.So I want to ...
user avatar
  • 11
0 votes
1 answer
12 views

One-sided time series trend-seasonal decomposition

TL;DR: Are there one-sided decomposition alternatives to the naive seasonal_decompose from statsmodels? Are there approaches to adapt intrinsically two-sided ...
user avatar
0 votes
0 answers
20 views

Music Recommander using Implicit Library

I want to build a music recommender predicting the number of times a user will listen to a song. I am using the Implicit library and following this close example : https://github.com/benfred/implicit/...
user avatar
0 votes
0 answers
14 views

How to overcome importerror for tensorflow

I'm working on ubuntu, and i'm trying to train my python model but i receive the following error ...
user avatar
  • 1
0 votes
1 answer
33 views

Classification Produces too Many False Positives or False Negatives

I trying to classify this data set (https://www.kaggle.com/datasets/fedesoriano/stroke-prediction-dataset) to classify if a patient is at risk for having a stroke. As the title says, whatever test I ...
user avatar
  • 3
0 votes
1 answer
18 views

Random Forest plot standardized

For a data science project, I first used a standardized scaler on data in python, ran random forest then plotted the tree. However, the values of the decisions are in their standardized form. How do I ...
user avatar
0 votes
1 answer
22 views

Traffic prediction using lstm

I am using LSTM model to predict the data traffic in every second of a base station. The dataset is as follows: The test and train prediction looks as follows: And the RMSE values for train score ...
user avatar
  • 1
0 votes
0 answers
22 views

python times series deconvolution value error BUG : filter coefficient a[0]==0 not supported yet

I have two time series (x, y) which have 852 length (same period). ...
user avatar
  • 1
0 votes
0 answers
10 views

Trying to implement a loss function read from a journal-article in python

Computer science undergrad here. I am trying to understand Eqn 12 from this paper so that I can implement it in python code. In this paper, the NN model takes a blurred image as input and outputs a ...
user avatar
1 vote
0 answers
10 views

Laben Encoding for Target Classes: Any Integer or Consecutive Integers from Zero?

I'm handling an very conventional supervised classification task with three (mutually exclusive) target categories (not ordinal ones): ...
user avatar
  • 7,917
1 vote
1 answer
28 views

How to return the number of values that has a specific count

I would like to find how many occurrences of a specific value count a column contains. For example, based on the data frame below, I want to find how many values in the ID column are repeated twice <...
user avatar
  • 15
0 votes
1 answer
18 views

Preprocessing in TensorFlow

Good night, I am working on a paper comparing Python libraries for machine learning and deep learning. Trying to evaluate Keras and TensorFlow separately, I'm looking for information about TensorFlow ...
user avatar
  • 3
0 votes
0 answers
10 views

Proper datashape and model architecture for recognizing highs and lows in a chart

I am using a Keras LSTM model to try to pinpoint the highs and lows (relative high points and low points) in a chart (I need the actual coordinates to those highs and lows, not just an image). The ...
user avatar
  • 101
0 votes
0 answers
29 views

Interpretation of SHAP summary plot in a multi class context

I'm performing multi-class classification and uses SHAP values to interpret the features. I have 3 classes. I have testet XGBoost and Multinomial Logistic Regression. When i'm using XGBoost I am able ...
user avatar
1 vote
1 answer
37 views

Is test data required to be transformed by training data statistics?

I am using a dataset (from literature) to build an MLP and classify real-world samples (from wetlab experiment) using this MLP. The performance of MLP on the literature dataset are well enough. I am ...
user avatar