Questions tagged [groupby]

A groupby operation: splits the data, applies a function to each group, and combines the results.

Filter by
Sorted by
Tagged with
1 vote
1 answer
27 views

How to impute and aggregate data with ID variant variables for predictive modeling?

I have a dataset that looks like so ID var_id_invariant_1 ... var_id_invariant_p var_id_variant_1 ... var_id_variant_k target 315 25 ... a 2.4 ... A 1 246 31 ... nan 5.7 ... B 0 315 25 ... a 9.4 .....
Mateusz's user avatar
  • 125
0 votes
0 answers
642 views

Aggregate a DataFrame using multiple conditions in groupby

I have a dataframe with many duplicates. I have determined rules for duplicate identification, for example : If street_name AND ...
Rusoiba's user avatar
  • 839
0 votes
0 answers
17 views

Formal terminology: metafeatures then groupby (GROUP BY) operation

In ML training and other analytics I often combine features to produce a 'metafeature' and then perform a 'groupby' (pandas) or 'GROUP BY' (SQL) query. What is the technical term for this operation? ...
M__'s user avatar
  • 208
0 votes
2 answers
89 views

Group rows partially [Python] [Pandas]

Good morning everyone. I have the following data: ...
Federica F.'s user avatar
0 votes
1 answer
20 views

Combination of groupby and mean methods

I am looking at the below csv file : We have the question : Display the mean of the variable gre by group of admitted/not admitted students, using the combination of groupby and mean methods. I would ...
evinda's user avatar
  • 101
0 votes
1 answer
5k views

How to groupby and sum values of only one column based on value of another column

I have a dataset that has the following columns: Category, Product, Launch_Year, and columns named 2010, 2011 and 2012. These 13 columns contain sales of the product in that year. The goal is to ...
MushyMush's user avatar
0 votes
1 answer
2k views

Using Pandas.groupby.agg with multiple columns and functions

I have a data frame which contains duplicates I'd like to combine based on 1 column (name). In half of the other columns I'd like to keep one value (as they should all be the same) whereas I'd like to ...
RobW's user avatar
  • 3
1 vote
1 answer
30 views

Grouping by Multi-Indices of both Row and Column [closed]

I have created a table using Pandas following material from here. The table created makes use of Multi-Indices for both columns and rows. I am trying to compute the descriptive statistics for each ...
Mar's user avatar
  • 21
0 votes
1 answer
28 views

Applied and view jobs ratio

I have the following data set where the column "kind" can be V(view) or A(apply), how can I do the following things given a particular job id how many applicant Apply (A) to that particular ...
GGT's user avatar
  • 115
0 votes
1 answer
3k views

How to find median/average values between data frames with slightly different columns?

I am trying to combat run-to-run variance of the data I collect by combining the data from different runs and finding the mean/average. The problem is that in each run there is a chance that some of ...
PhilipTsv's user avatar
0 votes
1 answer
58 views

panda grouping dates by variable with transposed value variables

I had read this post panda grouping by month with transpose and it gave me the nearest answer to my question but not the completely solution. How would I get somewhat like the reverse output? My ...
KevG.'s user avatar
  • 1
1 vote
1 answer
1k views

pandas groupby.count doesn't count zero occurrences

I am using groupby.count for 2 columns to get value occurrences under a class constraint. However, if value $x$ in feature never occurs with class $y$, then this pandas method returns only non-zero ...
Şafak's user avatar
  • 23
1 vote
0 answers
116 views

MongoDB Groupby Rank [closed]

Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
Noob's user avatar
  • 11
0 votes
1 answer
918 views

Using user defined function in groupby

I am trying to use the groupby functionality in order to do the following given this example dataframe: ...
Maurits van Roozendaal's user avatar
0 votes
0 answers
41 views

Accessing group elements in groupby

I had a doubt about groupby operations. Lets suppose I have grouped my data based on one column and got 5 groups as output. I know that we can iterate over these groups and apply functions to them as ...
Pari Ganjoo's user avatar
0 votes
1 answer
370 views

Access keys of pandas dataframe when using groupby

I have the following database: And I would like to know how many times a combination of BirthDate and Zipcode is repeated throughout the data table: Now, my question is: How can I access the keys of ...
Arnau's user avatar
  • 123
-1 votes
3 answers
5k views

Pandas Groupby datetime by multiple hours [closed]

I have a log dataset which looks like the following:- ...
WitchKingofAngmar's user avatar
0 votes
1 answer
2k views

Pandas - Avoid boolean result when using groupby()

I have this script: sectors = df.groupby(['company_sector']).mean()['investment_in_millions'] Output: I wanted to keep the same groupy() but having a result ...
Steven's user avatar
  • 119
1 vote
0 answers
48 views

Why is the behaviour of groupby command different in R and in python with pandas? [closed]

if I execute the following lines of codes in R as well as python for groupby command, the order of the resulting ...
Malathi's user avatar
  • 135
5 votes
1 answer
35k views

How to use df.groupby() to select and sum specific columns w/o pandas trimming total number of columns

I got Column1, Column2, Column3, Column4, Column5, Column6 I'd like to group Column1 and get the row sum of Column3,4 and 5 When I apply groupby() and get this that is correct but it's leaving out ...
Steven's user avatar
  • 119
1 vote
1 answer
258 views

pandas groupby and sort values

I am studying for an exam and encountered this problem from past worksheets: This is the data frame called 'contest' with granularity as each submission of question from each contestant in the math ...
JChang's user avatar
  • 13
1 vote
1 answer
2k views

Pandas - Sum of multiple specific columns [closed]

I created this script: ...
Steven's user avatar
  • 119
3 votes
0 answers
27 views

User defined aggregations on data of around 200GB where row order matters

I am working with "medium large" data of around 200GB. The data are long form log files, where there are several thousand logs for each "entity". The entities are actually flights ...
Placidia's user avatar
  • 226
1 vote
0 answers
74 views

Custom DataFrame format for exporting to excel sheets

I have the following DataFrame and I want it exported in an excel file with different sheets having different words as shown in the image ...
m2rik's user avatar
  • 321
0 votes
2 answers
527 views

Calculating Median and Mode of numerical variables for different subgroups in R

I have customer call data and I want to get the median and mode for the call success rate for different subgroups. My variables are: Customer ID, Employment Status (Retired, Employed, Unemployed), ...
ColRow's user avatar
  • 53
1 vote
1 answer
1k views

How to group by one column and count frequency from other column for each item in the previous column in python?

I am trying to group my data by the 'ID' column. Then I want to count the frequency of 'sequence' for each 'ID'. Here is a sample of the data frame: ...
Farah's user avatar
  • 21
0 votes
2 answers
614 views

How to combine rows after Pandas Groupby function

May I know how to combine several rows into one single row after I used Pandas groupby function? In below example, I would like to to group the data by Employee ID, Customer Last Name and Customer ...
Michelle's user avatar
2 votes
1 answer
784 views

Normalize data from different groups

I have data that has been grouped into 27 groups by different criteria. The reason for these groupings is to show that each group has different behavior. However, I would like to normalize everything ...
formicaman's user avatar
1 vote
1 answer
530 views

Group_by 2 variables and pivot_wider distribution based on 2 others

Performing some calculations on a dataframe and stuck trying to calculate a few percentages. Trying to append 3 additional columns added for %POS/NEG/NEU. E.g., the sum of amount col for all ...
DataGuy23's user avatar
1 vote
0 answers
20 views

Pushing down Group By clause [closed]

I've studied this exercise in class But I cannot figure out why, when I push down the Group By clause, I can remove RLCode attribute from GroupBy. Does this action change the meaning of query tree?
Lorenzoi's user avatar
0 votes
1 answer
330 views

Problem with sort by and group by in pandas

I have a DataFrame with duplicate id's but have different dates. For Example: ...
Shreya's user avatar
  • 1