Questions tagged [pandas]
pandas is a python library for Panel Data manipulation and analysis, e.g. multidimensional time series and cross-sectional data sets commonly found in statistics, experimental science results, econometrics, or finance.
0
votes
1answer
9 views
How to compare and find common values from different columns in same dataframe?
I would like to compare two columns and find common values sets in each column, then output the rows with the common values.
Let's say I have a dataframe with:
no.(col1) | Username (col2) | Referral(...
1
vote
1answer
9 views
2
votes
2answers
17 views
How do I find the count of a particular column, based on another column(date) using pandas?
I have a dataframe with 3 columns, such as SoldDate,Model and TotalSoldCount. How do I create a new column, 'CountSoldbyMonth' which will give the count of each of the many models sold monthly?
<...
0
votes
1answer
23 views
Merging multiple duplicate rows based on multiple columns in Pandas Dataframe [on hold]
I would like to merge nearly duplicate rows in a data frame like below:
...
1
vote
1answer
46 views
1
vote
0answers
23 views
How to write inputs append with time using panda in python
I have a data csv file include with three inputs temperature, humidity and wind. So in this csv file first input recorded at 6:00:00a.m.
But I want to start my time as 0 and then my second time is <...
0
votes
0answers
25 views
Looping through a list of pandas dataframes [closed]
First post so apologies in advance.
I have a list of DataFrames that were created with pandas.
I'm creating new columns in each DataFrame. Each new column is a calculated column from two of the ...
0
votes
0answers
35 views
how to write start_time= 0 and increment with 60 minutes to display y value (predict) continously using python
Here I have a dataset with three inputs, temperature, humidity, wind. I want to predict temperature value using these three inputs data in future at every 60 mins.
For the time code, here my first ...
1
vote
2answers
50 views
Obtaining consistent one-hot encoding of train / production data
I'm building an app that will require user input. Currently, on the training set, I run the following code, in which data is a pandas dataframe with a combination ...
0
votes
0answers
18 views
Pre-processing data to make predictions on deployed Sklearn model
I am new to Machine Learning. I have trained a ML model on the Diamond Prices Dataset to predict the price of a diamond given it's features (carat, cut color, clarity, etc...)
I have used pickle to ...
-3
votes
0answers
35 views
1
vote
1answer
26 views
How to find unique occurance of a value based an another value using pandas?
Below is my dataset.
...
1
vote
2answers
23 views
Around how much difference between two classes can be accounted as an imbalanced dataset?
I am building a binary classification model which has 17K values as class A and 10K values as class B. I want to know when a dataset can face the issues of "IMBALANCED Dataset" ?
1
vote
1answer
28 views
drop columns and rows in one line in pandas
I want to drop a range of rows and columns of a dataframe, I did it as follow:
...
0
votes
1answer
22 views
Converting pandas series object to int in pandas
I started my machine learning journey by deciding to explore recommender systems so that I can apply it in some of the projects for my company. Now, I am using Pandas for data analysis. The data set ...
0
votes
1answer
23 views
Python Script using pandas to plot histograms between the features
I am working around with data from Kaggle : Titanic Competition dataset.
Script
...
0
votes
1answer
24 views
-1
votes
0answers
24 views
I have to multiply the particular column with particular row by incresing number i.e; 1,2,3…upto n
I am using the code given below for multiplying the closing stock by 5*count:
...
1
vote
1answer
15 views
Read pandas data and convert it to IMMA
I'm working on a project in which my data set is in xarray and I need to write it to IMMA format. I converted it from xarray to pandas and am totally stumped on how to go from pandas to IMMA. Any ...
1
vote
1answer
22 views
how to iterate over certain rows of a certain column an update their value. by applying condition
I want to update the value of opening stock and closing stock by adding 20 when dcsDep2 < 3.028512.
here is the snapshot of dataset:
I tried with this code:
<...
0
votes
1answer
15 views
Transfer value of one column to another column into a new column based on condition
I have a dataframe with 2 relevant columns.
...
3
votes
1answer
45 views
How to automate the encoding process?
I am working on the Boston challenge hosted on Kaggle and I'm still refining my features. Looking at the dataset, I realize that some columns need to be encoded in binary, some encoded in decimals (...
4
votes
2answers
39 views
Which plot to use for data spanned on multiple years?
I've a dataset of - Number of accidents that happens in each state. The dataset spans from 2001 to 2014.
Which plot should I use to represent the information - number of accidents in each state per ...
3
votes
0answers
60 views
How to apply rolling volume profile function with a pandas TimeDelta
I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo.
Here the author gets slice the index and apply the main function by ...
-1
votes
0answers
31 views
Cohort analysis 3 months
I am struggling with a conditional groupby condition in pandas.
I have a dataset as it follows :
After grouping the data using the following code :
...
0
votes
0answers
16 views
Techniques to increase stability of parameters
I have a highly non-linear function $f(x,y,t)$, which I am trying to maximise over different time periods $t$. It is the case that other parameters are of mixed type with $x \in Re$ and $y \in {Z}$,
...
2
votes
2answers
224 views
one-hot-encoding categorical data gives error
I am currently working on the Boston problem hosted on Kaggle. The dataset is nothing like the Titanic dataset. There are many categorical columns and I'm trying to one-hot-encode these columns. I'...
1
vote
1answer
35 views
How to create feature representation?
Let's say I've a dataset with 800 rows(40 entries for each of 20 users). The entries are user session log ( columns are - browser, os, time, date etc for a specific session).
Now each user has unique ...
0
votes
1answer
21 views
Combine Pandas DataFrame Rows Based on Matching Data and Boolean
I have a Pandas DataFrame with sales data and columns for year, ISO week, price, ...
0
votes
1answer
22 views
Pandas dataframe groupby and then sum multi-columns sperately
I have a pandas dataframe as following:
...
2
votes
1answer
32 views
Histograms in Machine Learning
I have a large data set with over 100k samples and I want to predict a continuous target feature from 4 other continuous features using Scikit Learn.
For this project, I would like to visualize and ...
2
votes
0answers
21 views
Finding IDs that have been consistently failing to meet target
It's my first time here on stack exchange.
I have table with data columns including ID, Domain, Target and Dates (The table seen below). I am currently performing analysis on it and having trouble in ...
1
vote
0answers
27 views
2
votes
2answers
27 views
How to deal with millions or rows of data for analysis/visualization purpose [closed]
I have data in 2 tables in Sql server.
First table has around 10 million rows and 8 columns.
Second table has 6 million rows and 60 columns.
I want to import those tables into a Python notebook ...
2
votes
1answer
25 views
Change values of a particular column to value_count()
Suppose i have the following data set. I need to replace the value of a particular column by the value_count(). I saw few posts where it is done for the entire data set. I need to do it for a ...
1
vote
0answers
16 views
Finding Outliers in Resource Forecast Data
Unsure if this is the correct place to place this, please close if so.
I'm a workforce analyst at a large retail company, I own and maintain all the forecasting for our retail stores. This is based ...
0
votes
2answers
36 views
Removing duplicates and keeping the last entry in pandas
I have a table and I'm trying to remove all the duplicate and keep the
the rows that has the latest datestamp.
Here is the table:
...
0
votes
1answer
25 views
Extracting email id gives error
I am extracting email ids and storing them into a new column variable, but I am getting the issue:
...
0
votes
2answers
52 views
1
vote
0answers
22 views
How to create a heatmap inside a polar chart [closed]
I would like to build something like this. Creating a polar chart isn´t an issue, but i have no idea how to implement the round areas and the color gradients into the plot.
1
vote
0answers
24 views
Email clustering using numpy and pandas
Im trying to cluster my raw email data using numpy and pandas only.
Hence Im Trying to create an email dataframe from raw emails that I have. However the issue that I have is that I keep getting an ...
0
votes
0answers
43 views
Why modin.pandas takes more wall time than pandas?
All data scientist know that to scale the pandas to large dataset, we use modin.pandas instead of pandas. But when I tried to load a 500 MB csv dataset in my jupyter notebook in Ubuntu 16.04 to ...
0
votes
1answer
27 views
What is Difference Between np.zeros() and np.empty() [duplicate]
Can Anyone Please Explain me the Difference Between them In Terms Of Any Operation or Computation and Values Stored in them
0
votes
1answer
17 views
Script to convert non-numeric elements that are numbers to numeric
I'm self-teaching myself some scikit-learn and tensorflow right now, and trying to get better at cleaning data since the rest seems reasonably straightforward.
I downloaded some Google Play Store ...
1
vote
0answers
36 views
linear interpolation on python
Given 1000 objects, the data set tracks each object's location ordered by time, such as 3/3/18 9:01, 3/3/18 9:04.......how to define the first observation of each object together with xy-...
1
vote
1answer
110 views
1
vote
1answer
38 views
What is the difference between these 2 Python Pandas queries
I am trying to make sense out of some data, here is the Pandas data frame:
...
1
vote
1answer
10 views
Scatter Plot with Conditions
Suppose I have a data frame
name = ['A', 'B', 'C']
score = [2,4,6]
I want to create a scatter plot with the following conditions, color the bubble as green if ...
0
votes
0answers
5 views
I wanted to get the priority based on some column
I wanted to give priority based on values like in Col_2 there are 2 values A,B i Need A to be given more priority as compared to B incase if it has 2 values as per col_1
Col_1 Col_2 new_col
1 ...
2
votes
1answer
52 views
Processing csv file with more than 700K rows of data
I have a .csv file (around 400MB in size) which contains 700K rows of structured data.
Table structure is:
...
