Skip to main content
7 votes

How to group by multiple columns in dataframe using R and do aggregate function

...
Cabana's user avatar
  • 171
4 votes

How can I calculate mean and variance incrementally?

This problem was discussed, with proof and some alternate methods over on math.stackexchange.
Richmond Newman's user avatar
4 votes

How can I calculate mean and variance incrementally?

Following that link about moving variance in my comment, I came upon this: Welford's online algorithm for calculating variance, which seems to supply what I'm looking for. Here's the algorithm: <...
dubiousjim's user avatar
4 votes

How to group by multiple columns in dataframe using R and do aggregate function

I am sure there are better ways of doing it. Below is my simplistic take. ...
Drj's user avatar
  • 427
4 votes
Accepted

Aggregation of Discount

Aggregate discount sequences are used to define the stacking logic for aggregate discounts. This logic can define: The order in which aggregate discounts are applied. Which aggregate discounts are ...
dileep balineni's user avatar
3 votes

Pandas dataframe resample aggregation by mills too slow

It appears you don't really want to use resampling. You are immediately throwing away the resampled data. I think what you actually need is to simply groupby records in the same millisecond. That ...
Stephen Rauch's user avatar
  • 1,783
2 votes

How to group by multiple columns in dataframe using R and do aggregate function

I know this is an old post, but just thought I'd share my solution, which I think is a bit cleaner. ...
David Bloom's user avatar
2 votes

Privacy through moving averages?

The key question is how we can allow the public to make useful queries to a dataset without revealing private information. The field of Differential Privacy deals with answering just that. The key ...
mjul's user avatar
  • 307
2 votes

Learning from aggregated data

I would disagree with the statement that "training a model on aggregated data is harder than training a model on raw data". It is different, but I wouldn't say fundametally. It ultimately ...
Stefan Popov's user avatar
2 votes
Accepted

Pandas DataFrame: Aggregating multi-level groups by matching keys

Sorry, I realised all I needed to do was to add another level of grouping, but the aggregation is the same; ...
Steztric's user avatar
  • 181
2 votes

How to impute and aggregate data with ID variant variables for predictive modeling?

This sounds like you're having issues grappling with relational theory. You have focused on the ID column as though it identifies an observed example. But your narrative ("multiple services")...
J_H's user avatar
  • 1,065
1 vote

Learning from aggregated data

There are a few reasons why training a machine learning model on aggregated data might be different or harder than training on raw event data: Loss of Information: Aggregated data loses some of the ...
Vinita Silaparasetty's user avatar
1 vote

R-GUI How do i aggregate survey data collected for multiple years and see if they contain a variable?

If I understand your question correctly you'd like to obtain the proportion of missing values of a specific column in your dataset. This can be obtained with the plyr library For example: ...
Nicole's user avatar
  • 11
1 vote

How to aggregate the metrics from two different regression problems?

Indeed, you cannot aggregate because two problems may predict the price of stocks in dollars and the squared meters of houses, respectively. Your solution is feasible but cannot adapt continually; the ...
Eduard's user avatar
  • 669
1 vote

How to aggregate qualitative results from a simulation

If the features are considered categorical, think of the values as A,B,C,D. There's no possible mean value in this case, the most common way to aggregate would be to pick the mode, i.e. the value ...
Erwan's user avatar
  • 25.4k
1 vote
Accepted

Labeling and aggregating features issue

The goal is to predict if a user will subscribe in the future. By definition you cannot have labelled data now about what people will do in the future. However you can phrase the problem like this: ...
Erwan's user avatar
  • 25.4k
1 vote

How to get a (descriptive) overview of a large database?

There's no way to have a complete summary of a large dataset like this, you have to analyze what can be relevant, decompose into more specific pieces of information and then find the best way to ...
Erwan's user avatar
  • 25.4k
1 vote

How do I deal with data that has only limited target values?

I would try two different approaches: interpolate the missing values on a user level. work with the sunset of rows for which we actually have the glucose level. Then, I would compare the test ...
Guillermo Mosse's user avatar
1 vote

Getting the earliest date (duplicates due to several call ids and agents)

This solved the problem: select distinct on (customer) customer, agent, min(call) from call_data; I am using PostgreSQL.
ColRow's user avatar
  • 53
1 vote

Python Pandas agg error

If I am not mistaken the dataset used is the Boston home values dataset from http://lib.stat.cmu.edu/datasets/boston The given code works fine on google colab if the dataset used is correct. Please ...
Pragya Choudhary's user avatar
1 vote
Accepted

Can I apply survival analysis to predict if a user will revisit the website?

If you want to use survival analysis (which can be more flexible and insightful), I'd recommend this package and this great tutorial. Speaking shortly, as a result, you'll get "probability of being ...
i1bgv's user avatar
  • 116
1 vote

How can I get total sum of each group by using pandas

Assuming you have the above dataframe as df: ...
Sebastin Santy's user avatar
1 vote
Accepted

What are the approaches to aggregate categorical variables?

You problem is essentially you have high cardinality in your features, right? This will be relative to your problem, but you can look for mean encodings. Essentially, you will replace categories by ...
Victor Oliveira's user avatar
1 vote

Privacy through moving averages?

Depending on what you're working with here, one approach that might work would be to "jitter" the data. In other words: add noise. Your concern seems to be about people getting recognized from ...
David Marx's user avatar
  • 3,248
1 vote

R: Calculations based on frequencies / grouped / aggregate data

I think working with "collapsed" or "summarized" data as opposed to the data itself directly inside the SQL database is counter-intuitive. Also, trying to download the original table for analysis in R ...
FatihAkici's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible