Questions tagged [groupby]

A groupby operation: splits the data, applies a function to each group, and combines the results.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Approach to aggregating & creating subtotals for each new record / subgroup rather than doing a full "recalculation" over full dataframe each time

Just wondering if I could pick the brains of the many experts here. I am fairly new to programming in Python so please excuse the novice level question. I am currently working on a machine learning ...
0 votes
0 answers
33 views

Spark - Group by coulmn_1, and calculate the precentiles of coulmn_2 where the precentile to calculate is in coulmn_3

I have a data frame with 3 main columns. coulmn_1 - the column I want to preform group by on coulmn_2 - contains numeric values I would like to calculate percentiles on coulmn_3 - contains a number ...
  • 101
0 votes
0 answers
8 views

Analysis of Variance of different groups

Im working with the following dataset: ...
0 votes
1 answer
924 views

How to groupby and sum values of only one column based on value of another column

I have a dataset that has the following columns: Category, Product, Launch_Year, and columns named 2010, 2011 and 2012. These 13 columns contain sales of the product in that year. The goal is to ...
0 votes
1 answer
465 views

Using Pandas.groupby.agg with multiple columns and functions

I have a data frame which contains duplicates I'd like to combine based on 1 column (name). In half of the other columns I'd like to keep one value (as they should all be the same) whereas I'd like to ...
  • 3
1 vote
1 answer
26 views

Grouping by Multi-Indices of both Row and Column [closed]

I have created a table using Pandas following material from here. The table created makes use of Multi-Indices for both columns and rows. I am trying to compute the descriptive statistics for each ...
  • 11
0 votes
1 answer
24 views

Applied and view jobs ratio

I have the following data set where the column "kind" can be V(view) or A(apply), how can I do the following things given a particular job id how many applicant Apply (A) to that particular ...
  • 105
0 votes
1 answer
467 views

How to find median/average values between data frames with slightly different columns?

I am trying to combat run-to-run variance of the data I collect by combining the data from different runs and finding the mean/average. The problem is that in each run there is a chance that some of ...
0 votes
1 answer
28 views

panda grouping dates by variable with transposed value variables

I had read this post panda grouping by month with transpose and it gave me the nearest answer to my question but not the completely solution. How would I get somewhat like the reverse output? My ...
  • 1
1 vote
1 answer
603 views

pandas groupby.count doesn't count zero occurrences

I am using groupby.count for 2 columns to get value occurrences under a class constraint. However, if value $x$ in feature never occurs with class $y$, then this pandas method returns only non-zero ...
1 vote
0 answers
97 views

MongoDB Groupby Rank [closed]

Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
  • 11
0 votes
1 answer
382 views

Using user defined function in groupby

I am trying to use the groupby functionality in order to do the following given this example dataframe: ...
0 votes
0 answers
39 views

Accessing group elements in groupby

I had a doubt about groupby operations. Lets suppose I have grouped my data based on one column and got 5 groups as output. I know that we can iterate over these groups and apply functions to them as ...
0 votes
1 answer
61 views

Access keys of pandas dataframe when using groupby

I have the following database: And I would like to know how many times a combination of BirthDate and Zipcode is repeated throughout the data table: Now, my question is: How can I access the keys of ...
  • 113
-1 votes
3 answers
4k views

Pandas Groupby datetime by multiple hours [closed]

I have a log dataset which looks like the following:- ...
0 votes
1 answer
1k views

Pandas - Avoid boolean result when using groupby()

I have this script: sectors = df.groupby(['company_sector']).mean()['investment_in_millions'] Output: I wanted to keep the same groupy() but having a result ...
  • 109
1 vote
0 answers
27 views

Why is the behaviour of groupby command different in R and in python with pandas? [closed]

if I execute the following lines of codes in R as well as python for groupby command, the order of the resulting ...
  • 135
4 votes
1 answer
22k views

How to use df.groupby() to select and sum specific columns w/o pandas trimming total number of columns

I got Column1, Column2, Column3, Column4, Column5, Column6 I'd like to group Column1 and get the row sum of Column3,4 and 5 When I apply groupby() and get this that is correct but it's leaving out ...
  • 109
1 vote
1 answer
153 views

pandas groupby and sort values

I am studying for an exam and encountered this problem from past worksheets: This is the data frame called 'contest' with granularity as each submission of question from each contestant in the math ...
  • 13
1 vote
1 answer
1k views

Pandas - Sum of multiple specific columns [closed]

I created this script: ...
  • 109
3 votes
0 answers
24 views

User defined aggregations on data of around 200GB where row order matters

I am working with "medium large" data of around 200GB. The data are long form log files, where there are several thousand logs for each "entity". The entities are actually flights ...
  • 226
1 vote
0 answers
41 views

Custom DataFrame format for exporting to excel sheets

I have the following DataFrame and I want it exported in an excel file with different sheets having different words as shown in the image ...
  • 321
0 votes
2 answers
234 views

Calculating Median and Mode of numerical variables for different subgroups in R

I have customer call data and I want to get the median and mode for the call success rate for different subgroups. My variables are: Customer ID, Employment Status (Retired, Employed, Unemployed), ...
  • 53
1 vote
1 answer
480 views

How to group by one column and count frequency from other column for each item in the previous column in python?

I am trying to group my data by the 'ID' column. Then I want to count the frequency of 'sequence' for each 'ID'. Here is a sample of the data frame: ...
  • 21
0 votes
1 answer
382 views

How to combine rows after Pandas Groupby function

May I know how to combine several rows into one single row after I used Pandas groupby function? In below example, I would like to to group the data by Employee ID, Customer Last Name and Customer ...
2 votes
1 answer
428 views

Normalize data from different groups

I have data that has been grouped into 27 groups by different criteria. The reason for these groupings is to show that each group has different behavior. However, I would like to normalize everything ...
1 vote
1 answer
211 views

Group_by 2 variables and pivot_wider distribution based on 2 others

Performing some calculations on a dataframe and stuck trying to calculate a few percentages. Trying to append 3 additional columns added for %POS/NEG/NEU. E.g., the sum of amount col for all ...
1 vote
0 answers
19 views

Pushing down Group By clause [closed]

I've studied this exercise in class But I cannot figure out why, when I push down the Group By clause, I can remove RLCode attribute from GroupBy. Does this action change the meaning of query tree?
0 votes
1 answer
310 views

Problem with sort by and group by in pandas

I have a DataFrame with duplicate id's but have different dates. For Example: ...
  • 1