Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
1answer
30 views

How to convert duration column from 1 hr. 17 min to 77 min in pandas? [closed]

I am trying to convert all the hr., min, and sec into just mins. For example, 1 hr. 17 min to 77 min, and 34 sec to 0.56 min, not 0.34 min. So I have used this code: ...
0
votes
0answers
20 views

Time Series Analysis for Categorical Data Output

Suppose I am having dataset which consist of date as one column and fruits as second column which is categorical data having set of 4 different fruits in that column and my output column has 0's and 1'...
0
votes
0answers
36 views

Least mean square linear regression with discrete values on y-axis

So, the idea is that I have floating point numbers on x-axis and discrete values (colors) on y-axis. X-axis stands for measured values (temperature) and y-axis stands for predicted values (colors), ...
1
vote
0answers
18 views

Should I use Pad Sequence when using Word Vectors?

I have an unbalanced text data set. I want to use word vectors to embed words. When I use pad sequence? Before or after the word vector? I tried it, after the word vector I used pad sequence but my ...
1
vote
1answer
51 views

Imbalanced classification task – Discrepancy between learning curves and test set evaluation

I have a binary classification task related to customer churn for a bank. The dataset contains 10,000 instances and 11 features. The target variable is imbalanced (80% remained as customers (0), 20% ...
0
votes
0answers
20 views

spaCy text classification based on NER

I'm pretty new to spaCy and NLP in general, and I'm trying to figure out how to classify text. I've already gone through quite a few tutorials, and have figured out how to train my model, based on ...
0
votes
0answers
21 views

Image size for training transfer learning model for object detection purposes

I am trying to build transfer learning model to detect objects from video streams. There will be at least two or three different objects (classes) which are quite different from each other. The ...
0
votes
1answer
35 views

Select Random Value from Pandas list column for each row ensuring that value don't get picked again [closed]

I have a pandas DataFrame below ...
1
vote
1answer
40 views

ValueError: Error when checking input: expected time_distributed_6_input to have 5 dimensions, but got array with shape (32, 224, 224, 3) [closed]

I am trying to apply data augmentation to avoid overffiting in my CNN-LSTM image classification model. My training data has the shape: (1882, 1, 224, 224, 3) My ...
0
votes
0answers
29 views

A data in which an employee id is given in multiple months and many categorical features are there. To predict future retention. Recommend what to do?

I have this dataset in which we have to predict the retention of employees,i.e. how much will an employee stay in a company? This seems easy but the main obstruction here is that the same employee_id ...
0
votes
0answers
13 views

Is it possible to apply a time series generator to a image data generator?

I have the following code: ...
1
vote
1answer
31 views

Binary Classifier , when Data Points are very less and number of features are very large [closed]

I am building a Binary Classifier. There is no Real World Scenario Problem Statement, We have just given only the data set and some guidelines. Number of features : 2040 All features are in decimal ...
0
votes
1answer
27 views

Outlier treatment

I am working on a regression problem where I have a lot of outliers in multiple variables. As far as I can think of, there are 3 things I can do to outliers. Remove them (least attractive option) ...
0
votes
0answers
13 views

Why KNN output predicts all zeros rather one-hot label?

I have trained in supervised way several ML algorithms such as GNB,SVM, KNN. I have multi-class classification model (not multi-label). The input format has ~22 features and the output is one-hot ...
1
vote
0answers
22 views

How to use k_means algorithms on likert dataset?

I have a dataset of 10,000 customers with ten features all of which are Likert type. Like this: customer feature1 feature2 feature3 ID1 3 1 5 ID2 4 5 4 ID3 3 5 1 ID4 1 3 2 ID5 2 5 1 ID6 1 3 4 ...
0
votes
0answers
17 views

Visualization of multidimensional trajectories data stored as an array

I have 14-dimensional trajectories data in such form (an output of data.shape): (10000, 200, 14) and it looks like this: ...
1
vote
2answers
31 views

How to remove all empty elements from a list of list [closed]

I have the following list of list in Python and would like to remove all empty elements without invalidating indices. ...
0
votes
1answer
66 views

Should one log transform discrete numerical variables?

I am working on a Linear Regression problem and one of the assumptions of a Linear Regression model is that the features should be Normally Distributed. Hence to convert my non linear features to ...
1
vote
1answer
25 views

How to fill missing consumption data on time series?

I have a dataset that contains consumptions. These consumptions are measured every month. But some months are not measured. So the measured month after the unmeasured month is actually worth the sum ...
0
votes
1answer
52 views

How to convert horizontal bounding box coordinates to oriented bounding box coordinates

I am trying to detect oriented bounding boxes with faster rcnn for a long time, but I could not make it to do so. I aim to detect objects in the DOTA dataset. I was using built-in faster rcnn model in ...
0
votes
0answers
12 views

GIS Data on US College Dormitories

I am trying to find spatial data on all (or as many as possible) US college dormitories. Specifically, I want to find the location as a polygon or centroid. I have tried a few methods to acquire this ...
1
vote
0answers
26 views

How forecasting with LSTM model with Keras with event-based sample data? Python

I would like to do anomaly detection on a sensor, my data is recorded according to a change in value of +/- X%. From one value to another I can have a very short or very long time difference. The ...
0
votes
0answers
9 views

how to convert an array of non regularly interleaved coordinates to a matrix of weights using interpolation to obtain uniform sampling

I have an array of coordinates each one with an associated timestamp. Something like: ...
1
vote
0answers
16 views

Is there any library to perform robust clustering given two probability distribution with noise?

Given a dataset $X$ consisted with $w|X|$ samples drawn from a mixture of multivariate Gaussian distributions (say in two dimensions) and $(1-w)|X|$ samples of noise, is there any ...
0
votes
1answer
43 views

Cluster Evaluation with Jaccard and Rand Index

I've clusterized my data according to 3 criteria in 3 groups. I used kmeans to obtain those cluster so the label for each cluster is random and changes at each script run. To evaluate the consistency ...
3
votes
0answers
28 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
0
votes
0answers
100 views

How to track loss and accuracy in PyTorch?

I have made model and it is working fine for the MNIST dataset but further in the assignment it says to track loss and accuracy of the model, which I do not know how to do it. I have also written some ...
0
votes
1answer
136 views

Pandas series: how to compute daily cumulative sum of a column [closed]

Consider a pandas dataframe where each index is represented as a datetime object in the form like 2009-01-30 23:01:45 . In order to calculate the total value of a ...
1
vote
0answers
17 views

Does this ARIMA model take seasonality into account?

I'm writing a tutorial on traditional time series forecasting models. One key issue with ARIMA models is that they cannot model seasonal data. So, I wanted to get some seasonal data and show that the ...
1
vote
0answers
20 views

More training data - Less Memory

I have a training dataset of images common images, there are more than 5K images in this dataset. But I have less memory in Google colab- RAM-12GB. I need to train all the images but due to less ...
0
votes
0answers
17 views

Understanding ELMO Layers output_shape

I'm using the code below to use an ELMO layer and I need a little help understanding how the output_shape is used. Since the input to the model is a string (a full sentence), then is the output_shape ...
-1
votes
1answer
53 views

how to fill nan values based on conditions in pandas?

I am working on the Titanic dataset and there are null values in the Age column. I would like to fill them in based on there Pclass. For example, if class is 1, I ...
1
vote
1answer
34 views

How to solve a non classification problem with multiple plausible reults?(Tensorflow)

I'm fairly new to ML and now that I digged through tutorials and documentations I wanted to create a model myself now. The problem: I am a carpenter and back then in shool we had a problem where we ...
1
vote
0answers
18 views

Why is PyTorch's Dataloader is not inerrable?

I am working on MNIST dataset for an assignment and it seems to be I am stuck at some point for long. I have wrote my code for LogisticRegression and when I try to train the model it is not working as ...
0
votes
0answers
24 views

Training a DeepAR time series model with monthly data. Can only train on Daily

I am training a DeepAR model (arXiv) in Jupyter Notebook. I am following this tutorial. I create a collection of time series (concat_df), as needed by the DeepAR ...
0
votes
0answers
14 views

How to select the correct model for this data structure

This is a rough schema of the dataset I am working with. ...
1
vote
1answer
29 views

How to install custom pakages on Google-Colaboratory

I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'. I done the following things. First I cloned the git hub repository ...
0
votes
0answers
15 views

First-differencing non-stationary time series when fitting neural networks

I am trying out different types of neural networks for time-series forecasting and I have not been able to find a satisfying answer online about whether or not non-stationary series should be first-...
0
votes
0answers
17 views

How to predict categories without class labels using Bayesian methods?

Suppose I have the following financial data. I have to output a column with its result that shows what type of categories each row belongs. It can be an income, an expense, or a Capex (Capital ...
2
votes
1answer
53 views

Converting YYYYQQ to YYYY-MM-DD [closed]

I have been trying to convert stuff like 2002Q3 to 2002-09-01, but if I use the following, it will convert 2002Q3 to 2002-07-01: ...
0
votes
1answer
97 views

How to extract details (educational details, exp details etc.) from a resume?

I am trying to build a resume parser which can extract details such as Name, Address, Education details (degree name, college name, university name, course duration), Experience details (designation, ...
0
votes
2answers
64 views

Can I impute with median if median = 0?

I want to impute a numerical feature using the median, but the median for that feature is 0 and mean = 106. Should I go ahead and impute or is there anything else I can do? PS: I don't want to create ...
1
vote
1answer
22 views

Merging Multidimensional Features in Python

I am trying to do 10-fold cross-validation on an audio dataset. The audio is clipped into small segments, so we have multiple clips from the same file. To avoid overfitting, each audio is assigned to ...
2
votes
1answer
37 views

Efficiently finding most similar sports from large dataset

As part of a past interview task, I’m working with a sports streaming dataset that looks like this: ...
2
votes
2answers
155 views

Influence of imbalanced feature on prediction

I want to use XGB regression. the dataframe is coneptually similar to this table: ...
1
vote
1answer
71 views

Algorithms for casual feature selection for continuous Y

Currently I have been trying to find some good algorithms for feature selection. Using correlation or other non casual type of method will not be the right way to do a feature selection. I'm am ...
0
votes
1answer
60 views

How python LogisticRegression() binary classification works for more than 2 independent variables

I created bag of words (in which number of columns were around 15000 i.e greater than 1) and corresponding output data (0 or 1). After that I used LogisticRegression() (I didn't passed any parameters) ...
2
votes
4answers
99 views

Predicting Disease Drugs

I have a dataset in the format: ...
1
vote
0answers
24 views

Unable to get reproducible results with Keras using random seed

I am having problems in ensuring reproducibility of my Python script using Keras and random seeds. I have followed the guidelines provided here, but everytime I run the scripts I get different results....
1
vote
1answer
50 views

Best practices when creating a virtual environment in Anaconda

I am absolutely confused due to the number of different ways and the abundance of articles explaining how to create a virtual environment in Anaconda. You could use ...

1
3 4
5
6 7
120