Questions tagged [pandas]
pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.
708
questions
0
votes
1answer
17 views
I got the following error : 'DataFrame' object has no attribute 'data' can you help please
I do have the following error: AttributeError: 'DataFrame' object has no attribute 'feature_names'
appreciate your input
...
0
votes
1answer
22 views
empty fields to be filled with previous values
In the above sample data, I have empty fields and now the task is to fill the fields with previous values.
my columns are dates and the values are a number of items present for that particular article ...
0
votes
0answers
5 views
m = rating.pivot_table(index=['user_name'], columns=['site'], values = 'rating') [on hold]
m = rating.pivot_table(index=['user_name'], columns=['site'], values = 'rating')
I wrote this command, and it shows this error. I do not know what's wrong can any ...
1
vote
2answers
21 views
Optimization of pandas row iteration and summation
i'm wondering if anyone can provide some input on improving the speed and calculations of a pandas result.
What i am trying to obtain is a summation of IDs in one table (player table) based on each ...
0
votes
0answers
15 views
Deriving columns and filling them with existing data of the column values
hello guys need help in solving the problem, in the given image I have a column named as the filename which has more than 10000 rows basically there are CSV file names.
I want to create columns based ...
0
votes
0answers
12 views
Excel date-time auto fill error
I have data of 10 metrics for a 24 hour period which is in 5 minute intervals. I merged all the 10 data metrics into one file and converted it into a .xlsx file.
After opening the excel file I ...
0
votes
0answers
14 views
What types of variables are these? And what types of tests could I run?
So, I recently made a sheet of data collected from the 2010 Census on Cincinnati neighborhoods. The data is from demographics about the neighborhoods of the city such as median household income and ...
0
votes
1answer
11 views
From a mix of real-world and back-calculated data how to remove the part that was back-calculated?
I have a geostatistical dataset and I've been building linear regression model, but when I plotted the data I've noticed that part of the data shows an absolute straight line trend, i.e. it is most ...
0
votes
2answers
27 views
Is it a best practice to exclude retweets from the data set?
I am going to build machine learning algorithm to identify fake tweets. The data set has huge retweets which I think might be an issue. Do you think given that the focus is the original tweet, it is ...
0
votes
0answers
25 views
Reshape Time series data for Conv2d Block
I am modelling my time series data into a supervised learning problem for the input to a conv2d block in pytorch from this tutorial.
...
1
vote
2answers
30 views
Replace data in Pandas dataframe based on condition by locating index and replacing by the column's mode
Hopefully, you don't mind me posting a question here instead of the regular stack exchange forum. I'm learning ML basics and practising pandas.
assuming the data frame is called df, column name = ...
2
votes
2answers
33 views
Pandas datetime error when reading from excel file
I am trying to read an excel file that has two columns using pandas.
This is how the data looks in excel file:
...
0
votes
1answer
13 views
fast processing: changing the column value in a geospatial environment
I have a table whose header looks like this: complaint_type, borough, street_name, incident_zip, latitude, longitude
1) I want to check if the "incident_zip" column of each row is in a specific list ...
1
vote
3answers
109 views
creating a csv file from two csv files
I have two csv files:
sub_compiler.to_csv('sub_compiler.csv')
sub_compiler.head()
and
...
0
votes
1answer
26 views
Merging two pandas dataframes [duplicate]
I have two csv files,and if I do:
df_compiler = pd.read_csv('sub_compiler.csv')
I obtain:
and for the other csv file I do:
...
1
vote
1answer
35 views
Is there any way to create column based on some previous column in PANDAS dataframe?
Given: I have Pandas Dataframe as shown below
...
0
votes
1answer
12 views
error when submitting machine learning project
I am trying to make a submission of a machine learning classification problem. I have a test dataset where to try my model. To submit I have to build a csv file. The prblem is that when I go building ...
0
votes
1answer
43 views
'list' object has no attribute 'values' when we are using append in python
Here I have a dataset with three inputs. Here I generated y value using append. After the append I got the output like this:
y.append(rec.iloc[0]['y'])
Then I ...
0
votes
0answers
11 views
How to keep a list entry based on first appearance of a year? [migrated]
I have a list of lists:
I've been experimenting with different loops and for statements, but can't seem to get it to work. What I'd like to do is keep the entry based on first appearence of the year....
2
votes
1answer
33 views
1
vote
1answer
22 views
Changing the value of a column based on the condition on another column
I have a table whose header looks like this:
complaint_type borough street_name incident_zip latitude longitude
I want to check if the "incident_zip" column of each row is in a specific ...
0
votes
1answer
37 views
Making Sense of this Error Message
I am using a book and a video to learn how to use KNN method to classify movies according to their genres.This is my code:
...
0
votes
0answers
25 views
Viewing TensorFlow datasets vs. pandas DataFrames: Differences?
A simple preview of a pandas.DataFrame is typically done by calling head(k) and will simply return the first ...
0
votes
1answer
33 views
Replace entire columns in pandas dataframe
I would like to replace entire columns of a pandas dataframe with other columns, for example:
and I would like to replace the columns A and B. What I did is the following:
...
2
votes
1answer
30 views
Get elements from lists in pandas dataframe [closed]
I have the following column of a data frame:
I get it by doing dataFrame['opcodes].
and I would like to consider only the first 20 and the last 20 elements of ...
0
votes
1answer
17 views
Getting extra characters while loading data
While loading data using read_sas from sas dataset in Python getting b' along with every string. How can I fix this?
0
votes
2answers
34 views
use .apply() function to change values to a column of the dataframe
I have a dataframe which is the following:
and I would like to consider only the column of instructions and keep just the values push, test, mov, test ,....., so just the first word of each string ...
0
votes
0answers
30 views
pd.qcut bins error!
Hi Data Science community!
I'm working on some RFM analysis, but while setting up the bins, I'm getting an error. Here's my code and below the exact error.
...
2
votes
1answer
60 views
Why is pandas corr() deleting columns?
I'm doing a basic correlation analysis but for some reason pandas corr() is deleting columns, not sure why.
...
1
vote
1answer
34 views
Scikit model is not able to predict sequence correctly
I am trying to create a regression model using scikit-learn for predicting car price. The input data are, car model(trim), kilometers used, past resale price of similar car and age of used car. I am ...
0
votes
1answer
26 views
Problem building a feature vector
I am trying build a classifier for malware analysis for which basing in the instructions of an assembly code, such as push, mov,... I want to predict the compiler, and in a second time the ...
0
votes
2answers
76 views
AttributeError: 'DataFrame' object has no attribute 'c'
project=pd.read_csv("train.csv",sep=',')
this dataset train.csv has 200column, named as var_0,var_1 ....... var_199
...
1
vote
1answer
35 views
MemoryError when concatenating a large data-frame
I am creating a tool that reads CSV fields and allows the user to specify columns they want to categorize and then categorize those columns.
My issue is that these CSV files are quite large and when ...
0
votes
0answers
27 views
Connecting to IEX with Pandas Datareader
My Problem is regarding Algorythmic trading. I hope this is the right site for this kind of Question. In specific I try to connect to the "iex" API via the pandas Data reader to retrieve some ...
1
vote
0answers
30 views
Method for 'Simple' Clustering?
In a recent adult education class I took the prof shared a spreadsheet that was written in VBA code and takes ages to execute, I'm sure ...
0
votes
0answers
5 views
How to do cumulative sum over the last 30 days in Python? [migrated]
Apologies if there are similar questions answered online, but I wasn't able to adapt other code to mine to work successfully. My data looks like this:
...
0
votes
0answers
22 views
How to use groupby, resample, fillna fast with a large dataset
I have a time series of data in the following form:
...
0
votes
0answers
6 views
Pipeline that cached the results
I use pandas to do feature extraction for machine learning.
I hope to achieve the following: Consider I have five data processing steps done sequentially, and I execute it once, the results will be ...
-1
votes
1answer
50 views
2
votes
0answers
26 views
Need help on pandas data frames filtering on a csv file
Raw data gets generated daily
We need to get
success rate for current month on a daily basis &
success rate of each cell by end of that month from Jan till current month.
Raw data has following ...
0
votes
1answer
32 views
How to print nullity correlation matrix
I've a trainingset which has 400 features and most of them have null value.
I tried to draw the heatmap of nullity correlation matrix by means of Python and ...
0
votes
0answers
6 views
Averaging values across two different columns [migrated]
i have a dataframe as shown below. The elements in col_1 are connected to the elements in col_2, giving results in ...
1
vote
1answer
35 views
Numpy arithmetic operation between two columns
Below is the numpy array. I need to perform two operations on this array.
Add one column with value [column 1] - [column 3].
Add another column with value [column 1] - [previous value of column1].
I ...
0
votes
0answers
28 views
Pandas dataframe if else condition based on previous rows not working [migrated]
Is there any way to make the below code more efficient.
...
1
vote
0answers
18 views
Pandas nested/recursive groupby count [closed]
I have a DataFrame with more than 2 million rows, that looks like this:
...
0
votes
1answer
33 views
Find the most relevant columns for each single class in pandas
The following question (this one) did not help me.
I have a big dataset, and I want to know which Columns are the most relevant for the Target Variable. I know that, in my case, for each class in the ...
0
votes
3answers
77 views
Gradient decent in Python
I just finished working on my first machine learning algorithm i.e Linear regression. I want to reduce the rmse by optimising the model. I found out that gradient decent does the same job. But i dont ...
0
votes
1answer
46 views
Sublime Text 2 with Pandas for Excel (Combining Data) & Data Visualization
I'm new to programming with Python, and so far it's been a headache to create a build environment- and need your support and expertise in this area.
Background I'm running a MacBook, and using ...
0
votes
1answer
43 views
General approach on time series for customer retention/churn in retail
I have a time series of data in the following form:
...
0
votes
1answer
17 views
Interpreting DataFrame.where() documentation
From examples outside of the documentation, I thought I understood the examples of the .where() method. Basically, it seems to be a another way to filter a dataframe.
However, when I checked the ...
