Questions tagged [pandas]

pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.

Filter by
Sorted by
Tagged with
2
votes
2answers
557 views

'tuple' object is not callable while reshaping training data using python

I have data csv file with three inputs names temperature, humidity, wind. Here I want to predict temperature value in every 60 minute using LSTM model. Here I write the code to reshape the train . ...
0
votes
0answers
11 views

How to draw proper chart of distributional tree?

I am using python with matplotlib and need to visualize distribution percentage of sub-groups of an data set. imagine this tree: ...
1
vote
1answer
51 views

ValueError from statsmodels ExponentialSmoothing

I've been having a frustrating issue with the ExponentialSmoothing module from statsmodels. My data is a pandas series with 74 weekly data points that looks like this: ...
1
vote
1answer
42 views

Why is the difference of datetime = zero in the following dataframe?

This issue that I am facing is very simple yet weird and has troubled me to no end. I have a dataframe as follows : ...
1
vote
1answer
23 views

Converting a Pandas Series string of multiple attributes into individual attributes?

I'm working on an Airbnb dataset in order to predict the nightly price--where each example is one Airbnb listing (ie a listing for someone's house, room, apartment, etc). Each feature is a ...
0
votes
1answer
21 views

Index needs sorting help! [closed]

I want my index which is mixed up (i just sorted out ratings from highest to lowest) to be in ascending (sorted) order corresponding to my sorted ratings. How can I do that?
0
votes
1answer
40 views

How to find the mean of a column relative to another column?

I am working on the Boston house price prediction. I have a column named GarageYrBlt that holds the year the garage was built for a specific house. My assumption ...
0
votes
0answers
19 views

How to Manipulate date data for multiple visits per person?

I have a query to solve. I have data regarding customers and number of visits done to them. These are in two tables. So I want to join two table and create different features so that I can find better/...
0
votes
1answer
167 views

How to handle missing date data?

I have a column named GarageYrBlt which just lists the year the garage of that house was built. I have one nan value for this ...
0
votes
1answer
36 views

How do I find all the drastic changes in the stock and fetch news for those days of the stock?

I am analyzing MindTree company's stocks as an internship project using python pandas and have been given the task to find all the drastic changes by looking at the produced plot for the stock and ...
1
vote
0answers
31 views

How to write a code to display output value in every 60 minutes using panda python using csv file

I have temperature in a csv file. My file is updated with time . I have loaded this data to a pandas DataFrame. What I want to do is predict temperature value in next 60 minutes. I changed the start ...
0
votes
2answers
23 views

How to display the entire output and not shortened versions

I'm trying to display all the columns in my dataset that have NaN values: print(train.isnull().sum()) The output that I get is ...
0
votes
1answer
51 views
0
votes
2answers
126 views

How to use the fillna method in a for loop

I am working on a housing dataset. In a list of columns (Garage, Fireplace, etc), I have values called NA which just means that the particular house in question ...
0
votes
1answer
341 views

Dataset columns throwing KeyError

For some reason, the columns in my dataset are throwing the following KeyError: ...
0
votes
0answers
34 views

Error when checking input while running the LSTM model using panda python

I have a data set include with temperature, humidity and wind. Here I want to predict future temperature value in next hour. I used LSTM to predict future temperature value. But when I run the model ...
0
votes
1answer
922 views

How to remove outliers using box-plot?

I have data of a metric grouped date wise. I have plotted the data, now, how do I remove the values outside the range of the boxplot (outliers)? All the ['AVG'] data is in a single column, I need it ...
0
votes
2answers
95 views

What is the best way to replace NaN values for ranked columns

I have a column named BsmntQual that gives a ranking on the height of the basement per each house. These are all of the unique values in this column: ...
0
votes
1answer
313 views

Creating new column in dataframe based on conditions in 2 other columns [closed]

I would like to create a new column in my dataframe based on values from both the gender and experimental_grouping columns. As I have it written below, the column ...
1
vote
1answer
212 views

Python pandas index error

Encountred an index error when trying to run the code found here: https://github.com/snazrul1/PyRevolution/blob/master/Puzzles/DSP_For_Stock_Prices.ipynb The error occurs in section In[10]: ...
0
votes
1answer
35 views

Compare Rows Within a Group and Rank Best to Worst

I'm interested in an approach for comparing rows "within group" to produce a ranking from best to worst based on the performance relative to other rows within a group. For example, if given this ...
1
vote
2answers
2k views

Python to sum values in a column

I've created a Python code that reads the data from an excel file using Pandas. Code for your reference:- ...
0
votes
1answer
383 views

How to compare and find common values from different columns in same dataframe?

I would like to compare two columns and find common value sets in each column, then output the rows with the common values. Let's say I have a dataframe with: ...
0
votes
0answers
26 views

How to compare genetic profiles or vcf files in Python?

I have hundreds of vcf file where each vcf file contains genome profile for a tissue. A portion of the vcf file is as follows: I can read each vcf file into a dataframe. So it would be hundreds of ...
2
votes
2answers
32 views

How do I find the count of a particular column, based on another column(date) using pandas?

I have a dataframe with 3 columns, such as SoldDate,Model and TotalSoldCount. How do I create a new column, 'CountSoldbyMonth' which will give the count of each of the many models sold monthly? <...
1
vote
1answer
8k views

AttributeError: 'numpy.ndarray' object has no attribute 'columns'

...
0
votes
0answers
36 views

How to write inputs append with time using panda in python

I have a data csv file include with three inputs temperature, humidity and wind. So in this csv file first input recorded at 6:00:00a.m. But I want to start my time as 0 and then my second time is <...
2
votes
3answers
510 views

Obtaining consistent one-hot encoding of train / production data

I'm building an app that will require user input. Currently, on the training set, I run the following code, in which data is a pandas dataframe with a combination ...
0
votes
0answers
60 views

Pre-processing data to make predictions on deployed Sklearn model

I am new to Machine Learning. I have trained a ML model on the Diamond Prices Dataset to predict the price of a diamond given it's features (carat, cut color, clarity, etc...) I have used pickle to ...
1
vote
1answer
30 views

How to find unique occurance of a value based an another value using pandas?

Below is my dataset. ...
1
vote
2answers
38 views

Around how much difference between two classes can be accounted as an imbalanced dataset?

I am building a binary classification model which has 17K values as class A and 10K values as class B. I want to know when a dataset can face the issues of "IMBALANCED Dataset" ?
2
votes
1answer
101 views

drop columns and rows in one line in pandas

I want to drop a range of rows and columns of a dataframe, I did it as follow: ...
0
votes
1answer
114 views

Converting pandas series object to int in pandas

I started my machine learning journey by deciding to explore recommender systems so that I can apply it in some of the projects for my company. Now, I am using Pandas for data analysis. The data set ...
0
votes
1answer
38 views

Python Script using pandas to plot histograms between the features

I am working around with data from Kaggle : Titanic Competition dataset. Script ...
0
votes
1answer
961 views
1
vote
1answer
31 views

Read pandas data and convert it to IMMA

I'm working on a project in which my data set is in xarray and I need to write it to IMMA format. I converted it from xarray to pandas and am totally stumped on how to go from pandas to IMMA. Any ...
0
votes
1answer
143 views

Calling function on a list object

I have a list (sub_indices) that contains dataframe index values. Using these index values I need to make changes in other dataframe (main_df). Changes must be done on these index values that are in a ...
1
vote
1answer
64 views

how to iterate over certain rows of a certain column an update their value. by applying condition

I want to update the value of opening stock and closing stock by adding 20 when dcsDep2 < 3.028512. here is the snapshot of dataset: I tried with this code: <...
0
votes
1answer
41 views

Transfer value of one column to another column into a new column based on condition

I have a dataframe with 2 relevant columns. ...
4
votes
1answer
69 views

How to automate the encoding process?

I am working on the Boston challenge hosted on Kaggle and I'm still refining my features. Looking at the dataset, I realize that some columns need to be encoded in binary, some encoded in decimals (...
4
votes
2answers
50 views

Which plot to use for data spanned on multiple years?

I've a dataset of - Number of accidents that happens in each state. The dataset spans from 2001 to 2014. Which plot should I use to represent the information - number of accidents in each state per ...
3
votes
0answers
217 views

How to apply rolling volume profile function with a pandas TimeDelta

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo. Here the author gets slice the index and apply the main function by ...
0
votes
0answers
16 views

Techniques to increase stability of parameters

I have a highly non-linear function $f(x,y,t)$, which I am trying to maximise over different time periods $t$. It is the case that other parameters are of mixed type with $x \in Re$ and $y \in {Z}$, ...
3
votes
2answers
961 views

one-hot-encoding categorical data gives error

I am currently working on the Boston problem hosted on Kaggle. The dataset is nothing like the Titanic dataset. There are many categorical columns and I'm trying to one-hot-encode these columns. I'...
1
vote
1answer
45 views

How to create feature representation?

Let's say I've a dataset with 800 rows(40 entries for each of 20 users). The entries are user session log ( columns are - browser, os, time, date etc for a specific session). Now each user has unique ...
0
votes
1answer
122 views

Combine Pandas DataFrame Rows Based on Matching Data and Boolean

I have a Pandas DataFrame with sales data and columns for year, ISO week, price, ...
0
votes
1answer
86 views

Pandas dataframe groupby and then sum multi-columns sperately

I have a pandas dataframe as following: ...
2
votes
1answer
243 views

Histograms in Machine Learning

I have a large data set with over 100k samples and I want to predict a continuous target feature from 4 other continuous features using Scikit Learn. For this project, I would like to visualize and ...
2
votes
0answers
23 views

Finding IDs that have been consistently failing to meet target

It's my first time here on stack exchange. I have table with data columns including ID, Domain, Target and Dates (The table seen below). I am currently performing analysis on it and having trouble in ...