Questions tagged [aggregation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
0 answers
7 views

Learning the Average of a 0/1 Dependent Variable

uppose I have a matrix 𝑋 and a dependent vector 𝑦 whose entries are each in {0,1} dependent on the corresponding row of 𝑋 Given this dataset, I'd like to learn a model, so that given some other ...
user avatar
  • 1,147
0 votes
1 answer
33 views

How can we predict a value after several rows of data?

I have a regression problem in which for each week I have several rows (variable between rows i.e 1 week might have 1800 rows and other might have 5000 rows). My target is to predict a value at end of ...
user avatar
  • 1
0 votes
0 answers
18 views

Aggregating transactional data for customer segmentation

I have item-level transactional data where each row in the data represents a different item bought by a customer in a transaction (so if two different items were bought in the same transaction by the ...
user avatar
0 votes
1 answer
18 views

Tableau: keeping results independent of view / filter

I am using Tableau Desktop 2021.1.4 Suppose that my source sales data consists of 4 columns Region (dimension with values: N,E,W,S), Type (dimension with values: Furniture, Electronics, Appliances), ...
user avatar
  • 101
2 votes
1 answer
46 views

Labeling and aggregating features issue

I am trying build a simple binary classifier (some tree based algorithm for now) and my training data will have features aggregated at the user level. So I'll have a unique records of each user. These ...
user avatar
  • 123
0 votes
1 answer
17 views

Concatenating Data in two years

I have to use a Machine Learning Model to predict the Electricity consumption and carbon emission based on some buildings' features. (Area, year of construction ...) Here is the link to the data. The ...
user avatar
  • 101
0 votes
0 answers
18 views

How do you aggregate features of lists (pooling alternatives)?

Is it possible to reduce non-correlated multi-dimensional data over features to 1D data? A working option is pooling (mean/min/max) over an embedding vector (n samples of embeddings of m dimensions). ...
user avatar
  • 101
-1 votes
1 answer
55 views

How to aggregate data inserted by users to avoid outliers?

I'm developing a new application based on machine learning. In this application users can insert new data to improve the prediction system. As you may guess, users could insert data that doesn't make ...
user avatar
  • 1
1 vote
0 answers
77 views

MongoDB Groupby Rank [closed]

Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
user avatar
  • 11
0 votes
1 answer
149 views

Using R to organize/rearrange CSV - group by multiple columns?

I have a CSV that I need to clean up / organize in a usable way using R. I need to group by the property ID and then want to take all the unique years for the defor year column and make each year into ...
user avatar
1 vote
1 answer
22 views

How to get a (descriptive) overview of a large database?

I'm facing a data framework with ~ 20 k observations and 151 variables across 2078 subjects At first I am primarily interested in how the data looks like related to a single parameter. But I cannot ...
user avatar
  • 480
1 vote
0 answers
556 views

Heatmap of large 2D array using datashader and plotly

I’m trying to show a heatmap of a large 2D array (160x250000 entries). This should go into a dash app so I'm using plotly to deal with graphics and my idea was to use datashader for performance but I’...
user avatar
  • 56
1 vote
0 answers
21 views

Data system that manages aggregates over time intervals

I am looking to know if there is a data system that handles the following use case. To keep it simple, the data is a set of homogeneous enties E. E contains named numeric properties that the app code ...
user avatar
3 votes
0 answers
84 views

Difference Bagging and Bootstrap aggregating

Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron. Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
user avatar
  • 329
3 votes
0 answers
24 views

User defined aggregations on data of around 200GB where row order matters

I am working with "medium large" data of around 200GB. The data are long form log files, where there are several thousand logs for each "entity". The entities are actually flights ...
user avatar
  • 226
1 vote
1 answer
53 views

How do I deal with data that has only limited target values?

I'm currently working on a small project using the D1NAMO dataset (1). I want to predict the glucose level (that is given in the dataset) based on several features: accelerometer data, heartbeat (ECG) ...
user avatar
0 votes
1 answer
12 views

Getting the earliest date (duplicates due to several call ids and agents)

I am trying to get the earliest call interaction, with only the agent that interacted at that call and the customer id. Using the following code, I still get duplicates: ...
user avatar
  • 53
1 vote
2 answers
1k views

Python Pandas agg error

I am trying to generate descriptive statistics using agg function in Pandas. I am having trouble with one line with a lambda function. They work when I run them as separate lines of code, but when I ...
user avatar
  • 11
2 votes
1 answer
105 views

Feature Selection on Aggregated Targetdata

I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
user avatar
3 votes
1 answer
146 views

Can I apply survival analysis to predict if a user will revisit the website?

I have one business problem in hand which is to predict if a user will revisit the website or not within 6 months. I need to majorly understand what are the factors which make the user return and also ...
user avatar
  • 75
1 vote
1 answer
65 views

How do I deal with changing values in a categorical variable when I am aggregating customer records

My requirement is to build a model to predict if a new customer will return to their website. I need to analyze what drives customer repeat for both new and returning customers. The only information ...
user avatar
  • 75
0 votes
1 answer
27 views

How can I get total sum of each group by using pandas

I have a dataframe shaped like below ...
user avatar
  • 1
2 votes
1 answer
143 views

How to deal with a potencially multiple categorical variable

I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
user avatar
2 votes
1 answer
2k views

What are the approaches to aggregate categorical variables?

I am working on a clickstream dataset. I have come up with the following example dataset to explain my problem: ...
user avatar
  • 123
3 votes
1 answer
2k views

Pandas DataFrame: Aggregating multi-level groups by matching keys

I have some data that looks like this; ...
user avatar
  • 181
1 vote
2 answers
233 views

I want to be able to collapse and sum values dependent on the gene name

I have a table that looks like this: I want add together all the values for each gene for each column. For example, for LINC01128, it should read: ConN1 ConN2 ConN3 StN1 StN2 ...
user avatar
1 vote
3 answers
226 views

Data aggregation and split train test samples

I'm working on a data science project where the goal is to predict daily electricity consumption of a building based on some of its characteristics (e.g., size, location, etc.) and weather features (e....
user avatar
  • 15
1 vote
2 answers
390 views

Aggregating standard deviations

Imagine I have a collection of data, let's say the travel time for a road segment. On this collection I want to calculate the mean and the standard deviation. Nothing hard so far. Now imagine that ...
user avatar
  • 238
0 votes
1 answer
273 views

R-GUI How do i aggregate survey data collected for multiple years and see if they contain a variable?

I am new to R and usually rely on stata. I have a considerably large data frame, it contains data from a broad range of surveys and years, each with their own types of classifications and answers. ...
user avatar
  • 11
0 votes
1 answer
231 views

Ideal aggregation function for Partially Connected Neural Network (PCNN)

I am building a Python library that creates Partially Connected Neural Networks based on input and output data (X,Y). The basic gist is that the network graph is arbitrarily updated with nodes and ...
user avatar
2 votes
2 answers
52 views

Privacy through moving averages?

I am considering the following hypothetical situation: I have a time series of data. In general, 'the public' should have access to features of this data. However, making the time series available ...
user avatar
  • 131
1 vote
1 answer
59 views

Supervised learning on sources of information with different importance

I am trying to classify customer support sessions using supervised machine learning. In each customer support session I have 3 bags of information. 1. The title of the customer's complaint 2. ...
user avatar
  • 51
2 votes
2 answers
2k views

How can I calculate mean and variance incrementally?

Say I have a set S of values, and want to store in a database some summary information about that set, so that later when I acquire a new value v I can make a reasonable estimate of what the summary ...
user avatar
0 votes
1 answer
1k views

R: Calculations based on frequencies / grouped / aggregate data

I am trying to do simple calculations in R when no raw data but grouped data with frequencies is available only. This is the case when I have a large amount of records in a database, say a large SQL ...
user avatar
  • 11
4 votes
1 answer
3k views

Pandas dataframe resample aggregation by mills too slow

Given this test data: ...
user avatar
  • 41
3 votes
1 answer
230 views

Aggregation of Discount

I am trying to predict sales quantity of an item based on their attributes. Discount is one of those attributes. The problem is I am having different discounts in same period for same item .I need to ...
user avatar
2 votes
0 answers
67 views

Aggregating over sparse data

I am not sure if the title accurately reflects my problem but I essentially would like to aggregate a set of metrics of similar nature that comes from different data sources into a single metric. Say ...
user avatar
  • 131
0 votes
2 answers
41 views

Analytics term for turning row values into column names and count its assigned values

Do we have a data mining/analysis term for turning row values into column names and count its assigned values?
user avatar
6 votes
3 answers
142k views

How to group by multiple columns in dataframe using R and do aggregate function

I have a dataframe with columns as defined below. I have provided one set of example, similar to this I have many countries with loan amount and gender variables ...
user avatar
  • 1,025
4 votes
2 answers
140 views

Approach to creating a user profile in music web application

I am working on a use case, and I'm unsure of the best way to proceed: in order to analyze the behavior of users of a web-based music application, we retain all songs each has played since 2009. We ...
user avatar