Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Recurrent events analysis software (python)

I am looking for python packages suitable for analysis of recurrent events, preferably in the framework of non-linear Cox regression (which likely implies deep learning). Alternatively, tips for ...
0
votes
0answers
9 views

Multiclass ROC Curve using DecisionTreeClassifier

I built a DecisionTreeClassifier with custom parameters to try to understand what happens modifying them and how the final model classifies the instances of the iris dataset. Now My task is to create ...
0
votes
1answer
8 views

How to interpret this Plot of Model Loss from a BiLSTM model?

Hi everyone, the above graph is produced by a BiLSTM model i just trained and tested. I can't seem to interpret it while it is very different from the references that i acquired by googling online. ...
0
votes
1answer
18 views

Same confusion matrix when changing DecisionTreeClassifier parameters

I'm trying to build my first Decision Tree Classifier using the Iris dataset in the sklearn library. This is my first sample code: ...
0
votes
2answers
18 views

Is there a RandomForest implementation that handles categorical data without encoding in python?

I am working on a binary classification project with both continuous and categorical features. I know that the R implementation of RandomForest can handle categorical data passed in as factor type ...
0
votes
0answers
3 views

Understanding image size changes in DCGAN

I have been studying and trying to implement Generative Adversarial Networks using PyTorch. More precisely I tried to replicate the DCGAN PyTorch Tutorial tutorial using some custom dataset. My code ...
0
votes
0answers
9 views

ValueError : All input arrays (x) should have the same number of samples in LSTM [closed]

I have the following network where I want to use two different time windows for my lstm network, but got this error. Could you help me to solve that? ...
0
votes
0answers
12 views

How do I implement my loss function in Keras/Tensorflow, when it seems to have different parameters to the default ones?

So, I'm a university student studying Data Science, and after my previous question about TensorFlow got literally zero answers on Stack Overflow, I figured I'd post this one here instead. I need to ...
0
votes
0answers
13 views

Create a model that can extract only specific data out of receipts or invoices?

I'm trying to build a model that is capable of identifying only some of the information on receipts and invoices. All the documents having different structure in image format. Sample Data : Click here ...
1
vote
0answers
10 views

Cleaning GPS trajectory

I just want to get advice on how to clean GPS directory. Here is the scenario, though I can't give so much detail or data because of some rules. I have my dataset timestamp, latitude, and longitude. ...
0
votes
0answers
12 views

How is BCELoss counted in PyTorch? [different result comparing to mathematical implementation]

I am trying to understand how Binary Cross Entropy is counted in PyTorch. I've tried the same code from the PyTorch documentation here, but I get a different result comparing to mathematical ...
0
votes
0answers
14 views

NLP :Web Scrapping “News Articles” from financial website that included “MSFT” and “IBM” [closed]

I am planning to use BeautifulSoup to extract articles where the keyword "Microsoft" or "IBM" is mentioned. Preferred websites are: Google News Bloomberg New York Times Yahoo News ...
-1
votes
0answers
8 views

Read in file from Sharepoint folder on a given time (using Python) [closed]

I have a folder that lives within a sharepoint drive url. I wish to automatically extract a certain file, df, within this folder and import this file using Python. Desired Result: To schedule a 'job' ...
0
votes
1answer
21 views

Scaling of variables considering the values of a single column or the whole dataset

I read many time that for machine learning and data mining algorithms the multi-dimensional input data should be scaled (e.g. normalized or standardized). Now my question is whether the average, min ...
0
votes
0answers
9 views

Face matching using VGGFace library keras

I am working on face matching model (matching between id-card faces and selfies), where I am using the resnet50 pre-trained model from ...
1
vote
0answers
21 views

How to apply pruning on a BERT model?

I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
-1
votes
0answers
13 views

How to desplay changes over time in aerial images as heatmap? [closed]

I have number of aerial/satellite images of a forest taken over 10 years. I wanted to create a heatmap of changes over time of the area. I was able to find tools to create diff between 2 images, but ...
0
votes
1answer
27 views

How to choose between Tensorflow and Pytorch?

Recently I've been working on a pretty vanilla ANN model in Python with sklearn (and its preprocessing pipeline), mostly in jupyterhub notebooks if that matters. I am considering changing the ...
1
vote
1answer
20 views

Why does reducing the n_estimators in RandomForestClassifier improve accuracy? [closed]

I am taking a course that introduced me to sklearn.ensemble.RandomForestClassifier. At first it uses n_estimators with the default value of 10 and the resulting ...
1
vote
1answer
23 views

how print f1-score with scikit´s accuracy_score or accuracy of confusion_matrix?

I would like to print the f1-score. I got confused about the wording f1-accuracy score and accuracy score. What is the difference of these 2 scikit-learn metrics and how can I print the f1-score out ...
0
votes
0answers
14 views

multivariate LSTM for varying sequence length and varying Batch size

I receive data from a sensor and after some preprocessing, I get data points (1-18) with 8 features in every for-loop. That is, in one for-loop I get data of size 10x8 points while in the 2nd run, I ...
0
votes
1answer
14 views
2
votes
1answer
31 views

Correlation among features (e.g. doc length, punctuation, … ) in classifying spam emails

I extracted some other features from my dataset regarding punctuation, capital letters, upper case words. I got these value: looking at the correlation with my target variable (1=spam, 0=not spam), ...
4
votes
2answers
29 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...
0
votes
1answer
24 views

Looking for an advice based on an output of a LSTM model

I have fitted a LSTM model first using keras. Data:I have a time series with 560 observations. From that I trained the model using first 500 observations and then evaluated the model using last 60 ...
1
vote
0answers
12 views

Python : data type handling by sklearn and impact on memory usage and performance

I am currently working on reducing the memory usage of my data (Pandas DataFrame). This is going quite well : downcasting floats into smaller floats, integers into smaller ones and transforming string ...
0
votes
0answers
13 views

Separate weights for XGboost multi classification model? [closed]

I have a dataset that has 2 classes - 0 and 1. I have set the weights in the DMatrix to separate weights (as an array) for all the incorrect predictions. However, ...
0
votes
0answers
8 views

Python removing an already existing line when clicking on a chart to add a new line will remove the error bar top horizontal line on the first click [closed]

I am solving an exercise on coursera about making an interactive chart, where a user click on the chart, and an horizontal line is added, and based on its position colors of bars are changed. I just ...
2
votes
0answers
25 views

Precision-Recall Curve Intuition for Multi-Class Classification Utilizing SoftMax Activation [closed]

I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to ...
0
votes
0answers
12 views

Error using shap with SimpleRNN sequential model [closed]

In the code below, I import a saved sparse numpy matrix, created with python, densify it, add a masking, batchnorm and dense ouptput layer to a many to one SimpleRNN. The keras sequential model ...
-1
votes
0answers
15 views

Run python scripts on the cloud automatically [closed]

I need to run python scripts automatically on the cloud to stop depending on on-premise servers. This scripts would need to be run without any admin interference at least one time a day (they would ...
0
votes
0answers
6 views

Calculate values of a column of one based on data from another file [migrated]

I am newbie in python programming but I think it is the best language to program the solution to my problem. I need to multiply the values of a column of data based on the column name. The column data ...
2
votes
1answer
50 views

Questions about a multivariate timeseries forecasting model - keras

I have trouble understanding the model I'm trying to create. I have few questions so I'll explain my model first and what I'm trying to do: I have created sequences of data (input and ouput of the ...
0
votes
0answers
22 views

Image Classification with CNN - class_mode='categorical' [closed]

I am trying to use a CNN to classify Images. Here is my code: ...
0
votes
1answer
23 views

Algorithm for Multivariable timeseries prediction (COVID forecast)

I am trying to forecast tomorrow's COVID-19 cases in my country. I tried a simple Linear Regression implementation based on the "new_positives" field but it does not work very well. I had ...
0
votes
1answer
28 views

Using pretrained LSTM and Bert Models in CPU Only Environment - How to speed up Predictions?

I have trained two text classification models using GPU on Azure. The models are the following Bert (ktrain) Lstm Word2Vec (tensorflow) Exaples of the code can be found here: nlp I saved the models ...
0
votes
1answer
19 views

finding the mean for each of the channels (RGB) across an array of images [closed]

How can I find the mean for each of the channels (RGB) across an array of images? For example train_dataset[0]['image'].shape is ...
0
votes
0answers
25 views

Classification of tabular data with a CNN [closed]

I am trying to build a CNN to classify tabular data. For that purpose i use a base image to convolve the feature vectors of my dataset to create a new image based dataset. (There is a paper about that ...
1
vote
1answer
36 views

Python sklearn model.predict() gives me different results depending on the amount of data [closed]

I train my XGBoostClassifier(). If my testing set has: 0: 100 1: 884 It attempts to predict 210 1's. Around 147 are wrong (False positives) and 63 1's correctly ...
0
votes
1answer
13 views

Unable to successfully merge dataframes in pandas along labels

I have two different dataframes, they both share the same labels, "Country" and "Year", I am trying to merge these together as one by these two columns. This is my code: joined = ...
2
votes
1answer
47 views

Scikit-learn SelectKBest is picking up obviously unwanted Features

Dataset Dataset Summary: Bank Loan (classification) problem Problem Summary: I am exploring ways to simplify EDA Process (Exploratory Data Analysis) of finding the best fit variables I came across ...
0
votes
0answers
14 views

Should we convert year into dummies for machine learning? [closed]

I have a dataset on life expectancy. One of the column in dataset is year. Should I encode it with LabelEncoder to make linear regression model? I'm still learning machine learning. Please help.
-1
votes
0answers
13 views

How to cluster social media posts in the same category? [closed]

I am trying to build a clustering method that can group together the similar social media posts in the same categories. For example social media posts about promotional event or just random post. ...
0
votes
1answer
24 views

Model accuracy: how to determine it?

I have some doubts regarding the approach to build a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I have not a lot of experience in ...
3
votes
1answer
27 views

Smart sentence segmentation not splitting on abbreviations

Sentencer from SpaCy and NLTK does not catch the fact that typical abbreviations (e.g. Mio. for Million in German) and the ...
2
votes
1answer
192 views

Determining whether a Machine Learning model is overfitted with regard to the stability of the features

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect ...
0
votes
1answer
34 views

How to convert DataFrame column to Rows in Python?

I have the following dataset in df_1 which I want to convert into the format of df_2. In df_2...
0
votes
0answers
15 views

Extracting entities from Job Description (Each entity vs Combined Entity)? [closed]

I have corpus of job descriptions in raw text where each JD is appended as single long text. The ideal goal is to extract entities (like, skills, education, job title, certifications (if any)), I have ...
0
votes
2answers
21 views

Python Convert indexes starting from 1 to 365 into months names and display them into the x axis without changing the plot structure?

I need to display months on x axis of a plot instead of the indexes number of a data frame that goes from 1 to 365, which they represent the number of day of the year. So instead of an x-axis which ...

1
2 3 4 5
98