Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
3
votes
0answers
57 views

Find missing object(s) in image with a priori knowledge about the missing object(s) (w.r.t base image)

Problem Statement: I am working on developing a method, or borrow/modify/combine existing ones, where given an golden image (reference or base with all expected objects to be present), it is able to ...
0
votes
1answer
26 views

Predicting results of tennis matches based on historical data [closed]

I am writing my Bachelor thesis in Python about predicting results of tennis matches based on historical data. I have started from Logistic Regression but my model isn't efficient. If you could look ...
0
votes
0answers
30 views

ML Code throws value error when transforming data

Data source can be found here. I've hit a stumbling block in some code I'm writing because the fit_transform method continuously fails. It throws this error: ...
-1
votes
1answer
35 views

Model to choose with Cross Validation or not?

I made different tests on an imbalanced dataset and got these results: Model 1 = train test validation split + Cross Validation(cv=10) --> f1'micro' 0,95 Model 2 = train test split + smote method ...
0
votes
0answers
28 views

Different Results from Pre-Trained Model Between Python vs R

I am using a pre-trained Model. Firstly, I trained the Model in python which came with 94% accuracy. Now, I am trying to reproduce the same results with same configuration of the model in R, but it is ...
0
votes
2answers
23 views

Skip NA in Mean function within Pandas agg function [closed]

Here's my working code: new_df = old_df.groupby('store', as_index = False).agg({'reviewScore': 'mean'}) I would like to add an argument to the mean function: ...
1
vote
1answer
39 views

How to cluster user historical data? [closed]

I have transactional-level users data that includes their behaviour, such as reading articles, searching for content, posting, etc. I would like to cluster (most probably K-Means) these users based on ...
1
vote
1answer
14 views

what is criterion in flcuster of scipy package?

Could some one explain what does criterion of fcluster indicate? I tried to read the documentation but I am unable to understand. What does maxclust criterion indicate?
0
votes
1answer
24 views

Train and Test Data Split for a Tweet Classification Task

I am trying to train a few Machine Learning (ML) algorithms such as SVM, NB and Random Forest to do binary classification on disaster tweets. During this project, I want to train ML algorithms for a ...
0
votes
2answers
35 views

Pandas loads a csv file incorrectly, but without throwing an error [closed]

So I tried loading some data through pandas to practice manipulating it, but I ran into a slight problem. Basically, the pandas loads the data improperly. And it does so consistently. Let me show you ...
0
votes
1answer
22 views

How to normalize the data correctly in spam dataset

I'm working on the spam dataset to classify the inputs into binary classes. my problem is that: the observations in the dataset are floats small numbers in the first 53 column, and the 54 is float ...
0
votes
0answers
15 views

Recalculation the cluster centroids

I tried to cluster data points by k-medoids where k-centroids/medoids are randomly picked from (X), how those centroids are recalculated when there are some clusters changing by the updating x1? ...
0
votes
0answers
20 views

*(CATEGORIES[int(prediction[0][0])])* giving me different result for single image prediction from saved model

From a saved model, I am trying to predict a single image. I followed this code - https://www.youtube.com/watch?v=A4K6D_gx2Iw I am getting different result for two different command- ...
0
votes
1answer
33 views

Correctness of a ROC Curve

I've built a Decision Tree Classifier to practice with the sklearn library. My first task was to shuffle the iris dataset and split it keeping only the last 10 elements for the test. Then, after the ...
0
votes
1answer
23 views

How to explain MAE/MSE at each node of decision tree for regression in sklearn python? [closed]

If the mean value at any node is 60 and MSE = 169 so RMSE is 13. Can I conclude that the error at my node is 60 +-13 i.e my values in this particular sample split ranges from 60-13 to 60+13. If not , ...
0
votes
0answers
23 views

Multiclass ROC Curve using DecisionTreeClassifier

I built a DecisionTreeClassifier with custom parameters to try to understand what happens modifying them and how the final model classifies the instances of the iris dataset. Now My task is to create ...
0
votes
1answer
13 views

How to interpret this Plot of Model Loss from a BiLSTM model?

Hi everyone, the above graph is produced by a BiLSTM model i just trained and tested. I can't seem to interpret it while it is very different from the references that i acquired by googling online. ...
0
votes
1answer
24 views

Same confusion matrix when changing DecisionTreeClassifier parameters

I'm trying to build my first Decision Tree Classifier using the Iris dataset in the sklearn library. This is my first sample code: ...
0
votes
2answers
21 views

Is there a RandomForest implementation that handles categorical data without encoding in python?

I am working on a binary classification project with both continuous and categorical features. I know that the R implementation of RandomForest can handle categorical data passed in as factor type ...
0
votes
0answers
12 views

Understanding image size changes in DCGAN

I have been studying and trying to implement Generative Adversarial Networks using PyTorch. More precisely I tried to replicate the DCGAN PyTorch Tutorial tutorial using some custom dataset. My code ...
0
votes
0answers
26 views

How do I implement my loss function in Keras/Tensorflow, when it seems to have different parameters to the default ones?

So, I'm a university student studying Data Science, and after my previous question about TensorFlow got literally zero answers on Stack Overflow, I figured I'd post this one here instead. I need to ...
0
votes
0answers
19 views

Create a model that can extract only specific data out of receipts or invoices?

I'm trying to build a model that is capable of identifying only some of the information on receipts and invoices. All the documents having different structure in image format. Sample Data : Click here ...
1
vote
0answers
16 views

Cleaning GPS trajectory [closed]

I just want to get advice on how to clean GPS directory. Here is the scenario, though I can't give so much detail or data because of some rules. I have my dataset timestamp, latitude, and longitude. ...
0
votes
0answers
34 views

How is BCELoss counted in PyTorch? [different result comparing to mathematical implementation]

I am trying to understand how Binary Cross Entropy is counted in PyTorch. I've tried the same code from the PyTorch documentation here, but I get a different result comparing to mathematical ...
0
votes
1answer
28 views

Scaling of variables considering the values of a single column or the whole dataset

I read many time that for machine learning and data mining algorithms the multi-dimensional input data should be scaled (e.g. normalized or standardized). Now my question is whether the average, min ...
0
votes
0answers
14 views

Face matching using VGGFace library keras

I am working on face matching model (matching between id-card faces and selfies), where I am using the resnet50 pre-trained model from ...
1
vote
0answers
70 views

How to apply pruning on a BERT model?

I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
0
votes
1answer
38 views

How to choose between Tensorflow and Pytorch?

Recently I've been working on a pretty vanilla ANN model in Python with sklearn (and its preprocessing pipeline), mostly in jupyterhub notebooks if that matters. I am considering changing the ...
1
vote
1answer
22 views

Why does reducing the n_estimators in RandomForestClassifier improve accuracy? [closed]

I am taking a course that introduced me to sklearn.ensemble.RandomForestClassifier. At first it uses n_estimators with the default value of 10 and the resulting ...
1
vote
1answer
29 views

how print f1-score with scikit´s accuracy_score or accuracy of confusion_matrix?

I would like to print the f1-score. I got confused about the wording f1-accuracy score and accuracy score. What is the difference of these 2 scikit-learn metrics and how can I print the f1-score out ...
0
votes
0answers
20 views

multivariate LSTM for varying sequence length and varying Batch size

I receive data from a sensor and after some preprocessing, I get data points (1-18) with 8 features in every for-loop. That is, in one for-loop I get data of size 10x8 points while in the 2nd run, I ...
0
votes
1answer
25 views
2
votes
1answer
34 views

Correlation among features (e.g. doc length, punctuation, … ) in classifying spam emails

I extracted some other features from my dataset regarding punctuation, capital letters, upper case words. I got these value: looking at the correlation with my target variable (1=spam, 0=not spam), ...
4
votes
2answers
43 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...
0
votes
1answer
25 views

Looking for an advice based on an output of a LSTM model

I have fitted a LSTM model first using keras. Data:I have a time series with 560 observations. From that I trained the model using first 500 observations and then evaluated the model using last 60 ...
1
vote
0answers
14 views

Python : data type handling by sklearn and impact on memory usage and performance

I am currently working on reducing the memory usage of my data (Pandas DataFrame). This is going quite well : downcasting floats into smaller floats, integers into smaller ones and transforming string ...
2
votes
0answers
33 views

Precision-Recall Curve Intuition for Multi-Class Classification Utilizing SoftMax Activation [closed]

I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to ...
2
votes
1answer
53 views

Questions about a multivariate timeseries forecasting model - keras

I have trouble understanding the model I'm trying to create. I have few questions so I'll explain my model first and what I'm trying to do: I have created sequences of data (input and ouput of the ...
-1
votes
1answer
33 views

Algorithm for Multivariable timeseries prediction (COVID forecast)

I am trying to forecast tomorrow's COVID-19 cases in my country. I tried a simple Linear Regression implementation based on the "new_positives" field but it does not work very well. I had ...
0
votes
1answer
42 views

Using pretrained LSTM and Bert Models in CPU Only Environment - How to speed up Predictions?

I have trained two text classification models using GPU on Azure. The models are the following Bert (ktrain) Lstm Word2Vec (tensorflow) Exaples of the code can be found here: nlp I saved the models ...
0
votes
1answer
21 views

finding the mean for each of the channels (RGB) across an array of images [closed]

How can I find the mean for each of the channels (RGB) across an array of images? For example train_dataset[0]['image'].shape is ...
1
vote
1answer
38 views

Python sklearn model.predict() gives me different results depending on the amount of data [closed]

I train my XGBoostClassifier(). If my testing set has: 0: 100 1: 884 It attempts to predict 210 1's. Around 147 are wrong (False positives) and 63 1's correctly ...
-1
votes
1answer
26 views

Unable to successfully merge dataframes in pandas along labels

I have two different dataframes, they both share the same labels, "Country" and "Year", I am trying to merge these together as one by these two columns. This is my code: joined = ...
2
votes
1answer
62 views

Scikit-learn SelectKBest is picking up obviously unwanted Features

Dataset Dataset Summary: Bank Loan (classification) problem Problem Summary: I am exploring ways to simplify EDA Process (Exploratory Data Analysis) of finding the best fit variables I came across ...
0
votes
1answer
24 views

Model accuracy: how to determine it?

I have some doubts regarding the approach to build a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I have not a lot of experience in ...
3
votes
1answer
32 views

Smart sentence segmentation not splitting on abbreviations

Sentencer from SpaCy and NLTK does not catch the fact that typical abbreviations (e.g. Mio. for Million in German) and the ...
3
votes
1answer
210 views

Determining whether a Machine Learning model is overfitted with regard to the stability of the features

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect ...
0
votes
1answer
43 views

How to convert DataFrame column to Rows in Python?

I have the following dataset in df_1 which I want to convert into the format of df_2. In df_2...
0
votes
2answers
21 views

Python Convert indexes starting from 1 to 365 into months names and display them into the x axis without changing the plot structure?

I need to display months on x axis of a plot instead of the indexes number of a data frame that goes from 1 to 365, which they represent the number of day of the year. So instead of an x-axis which ...
1
vote
1answer
41 views

TicTacToe Linear Regression low accuracy and R^2 score

Im using the python sklearn library to attempt a linear regression TicTacToe AI. I create my training set by simply having the computer play random 'blind' games against itself. For example... Player ...

1 2
3
4 5
97