Questions tagged [aggregation]
The aggregation tag has no usage guidance.
48
questions
0
votes
0
answers
7
views
Record / subject wise aggregate classification
In the project I am studying I have a set of records (each record is a 1D signal) with variable length. For normalizing length and data augmentation, I am segmenting them in fixed length samples so I ...
0
votes
0
answers
83
views
Aggregate a DataFrame using multiple conditions in groupby
I have a dataframe with many duplicates. I have determined rules for duplicate identification, for example :
If street_name AND ...
- 819
1
vote
2
answers
164
views
Learning from aggregated data
Online and in the literature there seems to be a general consensus that training a machine learning model using aggregated data is harder and/or fundamentally different from training on raw event data....
- 120
0
votes
0
answers
9
views
is it sound practice to normalise & sum numerical variables to create a single metric for a prediction model? e.g's provided
Does this question aim to understand if this is sound practice? if not, I would appreciate a suggestion.
The goal is to use this metric:
regression
classification
as a metric
profile_views
...
0
votes
1
answer
28
views
How to aggregate the metrics from two different regression problems?
I'm about to conduct some tests to compare two solutions to regression problems. And to make the results more robust, I want to apply both on a few different datasets (all problems will be a ...
- 267
0
votes
0
answers
18
views
Finding Weekly Inventory from Sold Date and Inventory Added Date
I am aggregating data by week and need to find the number of items in inventory for each week. I have the date sold(Contract Date) and the date the item was added ...
0
votes
0
answers
41
views
Aggregating variables by two other variables in R using pivot_wider
I have data on cancer case counts in R that I would like to pivot from long to wide, such that a new count variable is created for each five-year age group within race/ethnicity groups. I have tried ...
1
vote
0
answers
26
views
Optimizing Rank Aggregation of Two Different Methods in Information Retrieval
As the title suggests, I would like to train a rank-aggregating model. My target problem is to rank text2s from a database as best as possible to a given query, <...
- 11
0
votes
0
answers
20
views
State of the art in transformer regressors/attention layer aggregation
I am curious to know what the state of the art is in using transformers for regression. Ultimately what I am interested in is how researchers in this field aggregate the outputs of the final attention ...
- 21
0
votes
1
answer
29
views
How to aggregate qualitative results from a simulation
I am trying to look for methods to aggregate and find the average of qualitative data that are outputted from a simulation.
There are 20 qualitative measures, each divided unevenly into 4 cycles ...
- 111
2
votes
0
answers
8
views
Learning the Average of a 0/1 Dependent Variable
uppose I have a matrix 𝑋 and a dependent vector 𝑦 whose entries are each
in {0,1}
dependent on the corresponding row of 𝑋
Given this dataset, I'd like to learn a model, so that given some other ...
- 1,197
0
votes
1
answer
95
views
How can we predict a value after several rows of data?
I have a regression problem in which for each week I have several rows (variable between rows i.e 1 week might have 1800 rows and other might have 5000 rows).
My target is to predict a value at end of ...
- 1
0
votes
1
answer
134
views
Tableau: keeping results independent of view / filter
I am using Tableau Desktop 2021.1.4
Suppose that my source sales data consists of 4 columns Region (dimension with values: N,E,W,S), Type (dimension with values: Furniture, Electronics, Appliances), ...
- 101
2
votes
1
answer
69
views
Labeling and aggregating features issue
I am trying build a simple binary classifier (some tree based algorithm for now) and my training data will have features aggregated at the user level. So I'll have a unique records of each user. These ...
- 123
0
votes
1
answer
21
views
Concatenating Data in two years
I have to use a Machine Learning Model to predict the Electricity consumption and carbon emission based on some buildings' features. (Area, year of construction ...) Here is the link to the data.
The ...
- 101
-1
votes
1
answer
84
views
How to aggregate data inserted by users to avoid outliers?
I'm developing a new application based on machine learning. In this application users can insert new data to improve the prediction system.
As you may guess, users could insert data that doesn't make ...
- 1
1
vote
0
answers
113
views
MongoDB Groupby Rank [closed]
Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
- 11
0
votes
1
answer
203
views
Using R to organize/rearrange CSV - group by multiple columns?
I have a CSV that I need to clean up / organize in a usable way using R. I need to group by the property ID and then want to take all the unique years for the defor year column and make each year into ...
1
vote
1
answer
38
views
How to get a (descriptive) overview of a large database?
I'm facing a data framework with
~ 20 k observations and
151 variables across
2078 subjects
At first I am primarily interested in how the data looks like related to a single parameter. But I cannot ...
- 510
1
vote
0
answers
962
views
Heatmap of large 2D array using datashader and plotly
I’m trying to show a heatmap of a large 2D array (160x250000 entries). This should go into a dash app so I'm using plotly to deal with graphics and my idea was to use datashader for performance but I’...
- 56
1
vote
0
answers
21
views
Data system that manages aggregates over time intervals
I am looking to know if there is a data system that handles the following use case.
To keep it simple, the data is a set of homogeneous enties E.
E contains named numeric properties that the app code ...
- 111
3
votes
0
answers
123
views
Difference Bagging and Bootstrap aggregating
Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron.
Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
- 329
3
votes
0
answers
27
views
User defined aggregations on data of around 200GB where row order matters
I am working with "medium large" data of around 200GB. The data are long form log files, where there are several thousand logs for each "entity". The entities are actually flights ...
- 226
1
vote
1
answer
60
views
How do I deal with data that has only limited target values?
I'm currently working on a small project using the D1NAMO dataset (1). I want to predict the glucose level (that is given in the dataset) based on several features: accelerometer data, heartbeat (ECG) ...
- 93
0
votes
1
answer
14
views
Getting the earliest date (duplicates due to several call ids and agents)
I am trying to get the earliest call interaction, with only the agent that interacted at that call and the customer id. Using the following code, I still get duplicates:
...
- 53
1
vote
2
answers
2k
views
Python Pandas agg error
I am trying to generate descriptive statistics using agg function in Pandas. I am having trouble with one line with a lambda function. They work when I run them as separate lines of code, but when I ...
- 11
2
votes
1
answer
173
views
Feature Selection on Aggregated Targetdata
I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
3
votes
1
answer
179
views
Can I apply survival analysis to predict if a user will revisit the website?
I have one business problem in hand which is to predict if a user will revisit the website or not within 6 months. I need to majorly understand what are the factors which make the user return and also ...
- 75
1
vote
1
answer
82
views
How do I deal with changing values in a categorical variable when I am aggregating customer records
My requirement is to build a model to predict if a new customer will return to their website. I need to analyze what drives customer repeat for both new and returning customers. The only information ...
- 75
0
votes
1
answer
33
views
2
votes
1
answer
186
views
How to deal with a potencially multiple categorical variable
I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
- 191
2
votes
1
answer
4k
views
What are the approaches to aggregate categorical variables?
I am working on a clickstream dataset. I have come up with the following example dataset to explain my problem:
...
- 123
3
votes
1
answer
2k
views
Pandas DataFrame: Aggregating multi-level groups by matching keys
I have some data that looks like this;
...
- 181
1
vote
2
answers
403
views
I want to be able to collapse and sum values dependent on the gene name
I have a table that looks like this:
I want add together all the values for each gene for each column. For example, for LINC01128, it should read:
ConN1 ConN2 ConN3 StN1 StN2 ...
1
vote
3
answers
318
views
Data aggregation and split train test samples
I'm working on a data science project where the goal is to predict daily electricity consumption of a building based on some of its characteristics (e.g., size, location, etc.) and weather features (e....
- 15
1
vote
3
answers
1k
views
Aggregating standard deviations
Imagine I have a collection of data, let's say the travel time for a road segment.
On this collection I want to calculate the mean and the standard deviation. Nothing hard so far.
Now imagine that ...
- 238
0
votes
2
answers
291
views
R-GUI How do i aggregate survey data collected for multiple years and see if they contain a variable?
I am new to R and usually rely on stata.
I have a considerably large data frame, it contains data from a broad range of surveys and years, each with their own types of classifications and answers. ...
- 11
0
votes
1
answer
251
views
Ideal aggregation function for Partially Connected Neural Network (PCNN)
I am building a Python library that creates Partially Connected Neural Networks based on input and output data (X,Y). The basic gist is that the network graph is arbitrarily updated with nodes and ...
2
votes
2
answers
59
views
Privacy through moving averages?
I am considering the following hypothetical situation: I have a time series of data. In general, 'the public' should have access to features of this data. However, making the time series available ...
- 131
1
vote
1
answer
66
views
Supervised learning on sources of information with different importance
I am trying to classify customer support sessions using supervised machine learning.
In each customer support session I have 3 bags of information.
1. The title of the customer's complaint
2. ...
- 51
4
votes
2
answers
2k
views
How can I calculate mean and variance incrementally?
Say I have a set S of values, and want to store in a database some summary information about that set, so that later when I acquire a new value v I can make a reasonable estimate of what the summary ...
- 181
0
votes
1
answer
2k
views
R: Calculations based on frequencies / grouped / aggregate data
I am trying to do simple calculations in R when no raw data but grouped data with frequencies is available only. This is the case when I have a large amount of records in a database, say a large SQL ...
- 11
4
votes
1
answer
3k
views
3
votes
1
answer
302
views
Aggregation of Discount
I am trying to predict sales quantity of an item based on their attributes. Discount is one of those attributes. The problem is I am having different discounts in same period for same item .I need to ...
- 137
2
votes
0
answers
97
views
Aggregating over sparse data
I am not sure if the title accurately reflects my problem but I essentially would like to aggregate a set of metrics of similar nature that comes from different data sources into a single metric. Say ...
- 131
0
votes
2
answers
44
views
Analytics term for turning row values into column names and count its assigned values
Do we have a data mining/analysis term for turning row values into column names and count its assigned values?
- 103
7
votes
4
answers
156k
views
How to group by multiple columns in dataframe using R and do aggregate function
I have a dataframe with columns as defined below. I have provided one set of example, similar to this I have many countries with loan amount and gender variables
...
- 1,045
4
votes
2
answers
145
views
Approach to creating a user profile in music web application
I am working on a use case, and I'm unsure of the best way to proceed: in order to analyze the behavior of users of a web-based music application, we retain all songs each has played since 2009. We ...
- 151