People who code: we want your input. Take the Survey

# Tag Info

4

Aggregate discount sequences are used to define the stacking logic for aggregate discounts. This logic can define: The order in which aggregate discounts are applied. Which aggregate discounts are mutually exclusive. Which aggregate discounts can be "stacked," or applied in conjunction with one another. You can use aggregate discounts within a quote or an ...

4

library(dplyr) library(tidyr) df %>% group_by(country, gender) %>% summarise(total_loan_amount =sum(loan_amount)) %>% spread(gender, total_loan_amount) %>% ungroup() %>% transmute(country = country, female_percent = F / (F+M), male_percent = M /(F+M)) results in Source: local data frame [1 x 3] ...

3

It appears you don't really want to use resampling. You are immediately throwing away the resampled data. I think what you actually need is to simply groupby records in the same millisecond. That can be accomplished with: Truncate to milliseconds and group by df['milliseconds'] = df['date'].str[:-3] grouped_and_summed = df.groupby(df.milliseconds).sum() ...

3

The first question is: what to do you want to see in user profile? Top-10 tracks, top-10 artist by user? How many tracks/artists a user listens to in a day on average (may be, in last month)? May be you want to get some general information related to the whole user base: Which artist/track is the most popular among users from different countries (top-N of ...

3

This problem was discussed, with proof and some alternate methods over on math.stackexchange.

3

I am sure there are better ways of doing it. Below is my simplistic take. library(dplyr); library(reshape2) Summary <- df %>% group_by(country, gender) %>% summarise(Net = sum(loan_amount)) final <- recast(Summary, country~gender, id.var = c("country", "gender")) final <- mutate(final, F_percent = final$F/(final$F+final\$M), M_percent ...

2

I know this is an old post, but just thought I'd share my solution, which I think is a bit cleaner. library(tidyverse) df <- data.frame(stringsAsFactors=FALSE, country = c("Austia", "Austia", "Austia", "Austia", "Austia", "Austia"), loan_amount = c(175, 100, 825, 175, 1025, 225), gender = c("F", "F", "M", "F", "M", "F") ) df %>% ...

2

You can download a free as in beer software Qlikview that allows you to do interactive data discovery via graphical interface similar to Excel but also featuring a powerful scripting language for data load and transformation. Huge flat files is no problem at all. It is an in-memory technology so you'd need a computer with a lot of RAM. The advantage though ...

2

The key question is how we can allow the public to make useful queries to a dataset without revealing private information. The field of Differential Privacy deals with answering just that. The key concept is to track a user's queries and the information revealed and add uncertainty to the answers to guarantee privacy. For example, a broad query like "what ...

2

Sorry, I realised all I needed to do was to add another level of grouping, but the aggregation is the same; stockBinGroups = data.groupby(['stock', 'binNum'], sort=False) binAveVol = stockBinGroups.aggregate({'volume': np.mean}).reset_index() binAveVol.head() stock binNum volume 0 stock0 2 174095.238095 1 stock0 3 100428.571429 ...

1

There's no way to have a complete summary of a large dataset like this, you have to analyze what can be relevant, decompose into more specific pieces of information and then find the best way to visualize each specific part on its own. The first thing would be to plot the distribution of this parameter of interest across subjects and/or observations. If you ...

1

I would try two different approaches: interpolate the missing values on a user level. work with the sunset of rows for which we actually have the glucose level. Then, I would compare the test accuracy of the model built with both methods. Remember that your test set has to be composed of rows for which you have the glucose level - you cannot build it with ...

1

This solved the problem: select distinct on (customer) customer, agent, min(call) from call_data; I am using PostgreSQL.

1

If you want to use survival analysis (which can be more flexible and insightful), I'd recommend this package and this great tutorial. Speaking shortly, as a result, you'll get "probability of being alive" for each customer. If you want to use logistic regression I think it's trickier. Why I think so - Like any other churn problem, it's hard to define it ...

1

Assuming you have the above dataframe as df: df.weight = df.weight.astype(int) df.weighted_hour = df.weighted_hour.astype(int) df["Total_WH"] = df["weight"] + df["weighted_hour"] df1 = df.groupby(['size'], as_index=False).sum().assign(ind='Total') df = pd.concat([df, df1]).sort_values('size')

1

You problem is essentially you have high cardinality in your features, right? This will be relative to your problem, but you can look for mean encodings. Essentially, you will replace categories by the mean on target variable, however, this is highly prone to overfitting and you should take care. The following two videos will give an excellent explanation: ...

1

If I understand your question correctly you'd like to obtain the proportion of missing values of a specific column in your dataset. This can be obtained with the plyr library For example: library(plyr) df <- data.frame(Study=c('Study_1', 'Study_1', 'Study_2', 'Study_3'), col1=c(NA,NA,5,8), col2=c(8, NA, NA, 7)) column <- 'col1' # is the ...

1

Depending on what you're working with here, one approach that might work would be to "jitter" the data. In other words: add noise. Your concern seems to be about people getting recognized from specific timestamp values, so jitter those. depending on the data, you could end up with new timestamps that don't align with any individuals in the data at all while ...

1

Following that link about moving variance in my comment, I came upon this: Welford's online algorithm for calculating variance, which seems to supply what I'm looking for. Here's the algorithm: new_count = old_count + 1 d1 = new_value - old_mean new_mean = old_mean + d1/new_count d2 = new_value - new_mean new_sum_squares = old_sum_squares + d1*d2 From a ...

1

I think working with "collapsed" or "summarized" data as opposed to the data itself directly inside the SQL database is counter-intuitive. Also, trying to download the original table for analysis in R is a very bad route if the data is too big to fit in the memory, and given that you already have a tool like SQL, which is designed for that specific purpose. ...

Only top voted, non community-wiki answers of a minimum length are eligible