Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
74 views

How to interpret pairplot?

There are 2 sns.pairplot, tell me how to interpret them. As I understand it, sns.pairplot allows us to look at the diagonal distribution of these signs, and on the non-diagonal linear relationship ...
0
votes
0answers
144 views

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0,

I am new to python and programming for that matter. I am trying to solve a special type of wave equation using finite element method and carry out simulations in python. I am getting some errors which ...
0
votes
1answer
48 views

Using LSTM for multi label classification

I am trying to use LSTMs to train and predict authors using reviews data and metadata ...
1
vote
1answer
36 views

Attribute error when loading pickle file in python jupyter notebook

I have created a config file with the following code for an object detection task and saved in the local disk. ...
0
votes
2answers
35 views

Is autocorrelation the same as multicollinearity?

I have a time series dataset that has 63 features with traffic_volume as the target. ...
0
votes
0answers
23 views

How to use groupby, resample, fillna fast with a large dataset

I have a time series of data in the following form: ...
0
votes
0answers
9 views

Predicting next word embeddings vector with LSTM

I'm tring to take parsed log data from, e.g. OpenStack, where one line looks for example like this: ...
0
votes
0answers
16 views

Workaround on using Grid Search when we have scipy.sparse.csr.csr_matrix

I am reviewing some techniques based on scikit-learn and I would like to check what are the best parameters for SVM using Grid Search. The thing is that I don't know how to use Grid Search to the ...
0
votes
0answers
13 views

Combining two structurally similar datasets from different sources

I am working with a text summarization problem, and I am trying to use this architecture [Pointer Generator]. My data set is VERY small (225 samples) compared to the CNN/ Daily Mail dataset this paper ...
1
vote
1answer
34 views

What to do with large number of collinear variables?

I have this time-series dataset that has 63 features, out of which 57 were manually engineered. While checking for collinearity, I get this correlation matrix: As can be seen there are a number of ...
0
votes
1answer
26 views

LSTM - Incorporate word embedding in layer with multiple records in same date

I have a time series data having more than one record in a single date. Number of records in a single date is not consistent. I have 3 input features namely phrase, cost and weight. My goal is to ...
0
votes
0answers
12 views

Data input and shape for LSTM

I want to classify data with a LSTM. I need a binary classification, thus 2 classes (0,1). I probably need a stateful LSTM as one CSV file (sequence) is a continous recording in time. The data is ...
0
votes
0answers
13 views

What language is most used in deep learning research?

I see the GAN source code all in Python, or otherwise, BERT seems to be all in Python as well. I was under the impression that ...
0
votes
0answers
7 views

Meaning of hidden states in a recurrent neural network

What does the term "state" in terms of "hidden state" mean in recurrent neural networks. Does it imply memory?
0
votes
0answers
28 views

Convert column of float valus to time format using panda

I have a data frame with a column of float data (e.g. 1.15 or 12.50).The first number represent hour and the digit represent minutes. I want to format it in order to have 4 numbers by adding 0 to ...
-1
votes
1answer
71 views
0
votes
3answers
35 views

What is the better to pre-process data, MS-Excel or Python/R [closed]

While handling data, many anomalies occur that might change the results. For example, outliers, missing values, etc. So in this context, what is the better tool to pre process data? A visual tool like ...
2
votes
0answers
28 views

Need help on pandas data frames filtering on a csv file

Raw data gets generated daily We need to get success rate for current month on a daily basis & success rate of each cell by end of that month from Jan till current month. Raw data has following ...
0
votes
0answers
43 views

Data Augmentation Multi Outputs

This question is asked several times here on SE, but I havent been able to find the right answer. I'm trying to build a network with 1 input and 2 outputs. I don't have a lot of data so I would like ...
0
votes
0answers
30 views

Executing python in R-markdown using reticulate package

I want to use pandas profiling in R-markdown but the code is not working ! ...
0
votes
1answer
33 views

How to print nullity correlation matrix

I've a trainingset which has 400 features and most of them have null value. I tried to draw the heatmap of nullity correlation matrix by means of Python and ...
1
vote
0answers
8 views

How to create a seq2seq without specifying a fixed decoder length?

Based on the model presented in this answer: ...
0
votes
0answers
33 views

Empty Dataframe, Columns [], Index [], not sure how to resolve

The goal for this application is to read stock tickers from excel to python, The information reads and prints great, however when I run the program that reads this 'Tickers' data frame, I obtain this: ...
0
votes
1answer
21 views

Extracting MM-YYYY from python date and creating a new column with the same [closed]

I want to extract the month and year from one of my date columns in the dataset and create a new column in the data-frame with the new MM-YYY format. My current solution is working fine but its way ...
0
votes
1answer
23 views

extract document topic vectors from lda model

how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation
2
votes
1answer
26 views

What is the best practice for data folder structuring?

I work for a small data science consultancy firm and we are trying to standardize our project folder structure. We started from the cookiecutter structure which is a great base. However one of the ...
1
vote
1answer
63 views

LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
0
votes
0answers
86 views

Regression on list of 2D time-based data

I have data such that: $D=\{(X_1, Y_1, z_1), (X_2, Y_2, z_2), ...,(X_n, Y_n, z_n)\}, n=1000$ Where: $|(X, Y)_k|$ varies in the $[10, 10000]$ range; $x$ are time values; $y$ are values in $T=\{1, 2, ...
0
votes
0answers
7 views

TimeDistribution GRU with initial_state keras

I am working on a problem, where we have two inputs, a GRU layer, and an output layer: ...
0
votes
0answers
12 views

regarding 3D scatterplot with line in Python or Matplotlib

In Matplotlib or other python library, how to generate the 3D scatter plot with line, shown as following? It seems that ...
0
votes
2answers
32 views

Validate via predict() or via fit()?

There are several possibilites to evaluate a model: hist = model.fit(x_train, y_train, (...) validation_data=(x_test, y_test)) or to use <...
1
vote
1answer
36 views

Numpy arithmetic operation between two columns

Below is the numpy array. I need to perform two operations on this array. Add one column with value [column 1] - [column 3]. Add another column with value [column 1] - [previous value of column1]. I ...
3
votes
2answers
28 views

Classifying dates in sentences

Let's say I have a sentences that goes like this: Hi, how are the kids. I will be going to Los Angeles next Friday and will come back the following Monday. If the date today is October 16 (...
0
votes
1answer
26 views

How to fix TypeError in spyder?

How to fix TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''? ...
1
vote
1answer
27 views

How to generate abbrevations from shortend words in medical records

I have text files which contains medical history of a patients and would want to extract information out of it. Basically what want is generate english text of abbrevations,semantic category region,...
0
votes
1answer
34 views

Linear regression doesn't return the expected number of $\beta_i$

I have a dataset of precincts and results of parties on different elections. After reading this article I really wanted to use linear regression to answer the question : how did voters changed their ...
0
votes
2answers
42 views
1
vote
0answers
34 views

Multilabel Classification; which network design?

I have a hard time thinking about how I can build this network with the following problem: I want to build a CNN to classify notes from sheet music. I have tried several models with and without ...
1
vote
0answers
19 views

Pandas nested/recursive groupby count [closed]

I have a DataFrame with more than 2 million rows, that looks like this: ...
0
votes
0answers
20 views

ResNet50 Model is not learning with transfer learning in keras

I am trying to perform transfer learning on ResNet50 model pretrained on Imagenet weights for PASCAL VOC 2012 dataset. As it is a multi label dataset, I am using ...
1
vote
1answer
21 views

Error rate of AdaBoost weak learner always bigger than 0.5?

As far as i understand, weak learners of AdaBoost should never yield a error rate > 0.5 After training one, i only receive error rates above 0.5. How is that even possible? The AdaBoost Tree still ...
0
votes
0answers
13 views

evaluate pretrained model on a new class and previously learned classes

I am trying to write a model to demonstrate catastrophic forgetting. I am using cifar10 image dataset that has got images from 10 different classes. Firstly, I trained the model on 8 classes (except <...
0
votes
1answer
34 views

Keras - Autoencoder different from Encoder + Decoder

I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
0
votes
1answer
55 views

“ValueError: Index contains duplicate entries, cannot reshape” error when I try to use pd.MultiIndex.arrays

I have data which includes id , gender , collected time test name and Test values , Units of measurement Test Names will include all tests that a patient taken and Value col will have its ...
1
vote
0answers
17 views

How to generate sports tracking data using deep learning?

Data: I have a 2D Numpy array that contains tracking data for football. Each row has the (x,y) coordinates for all players + the ball. That's 22 players and 1 ball = 46 columns. The frequency is 0.1 ...
0
votes
0answers
8 views

How can I classify single fused gray scale image in python ? is it possible to binary classsify single output image with Ground Truth image?

I want to calculate the precision, recall, and accuracy of the single predicted image(y_pred) with the Ground truth (y_true) image. I have only two binary class (0 and 1) so my question is, ** is it ...
0
votes
0answers
16 views

Running python script into SAS EG

I need some help running a python script from inside SAS. I want to take advantage of the possibility of scheduling a SAS code to run everyday, i already have a script in python that manipulates ...
0
votes
0answers
6 views

Retrieving range on axis based on the peak point in scatter plot

I have a scatter plot that shows data of products sold & sales for each store. As visible in the graph that there is spike in sales from 60-70. I need a way to extract this range. Since i need to ...
0
votes
1answer
30 views

Predicting churn - deal with missing dates in time series and improve modelling result

This is the follow up question for General approach on time series for customer retention/churn in retail. I have a time series of data in the following form: ...