Questions tagged [r]

R is a free, open-source programming language and software environment for statistical computing, bioinformatics, and graphics.

Filter by
Sorted by
Tagged with
107
votes
15answers
109k views

Python vs R for machine learning

I'm just starting to develop a machine learning application for academic purposes. I'm currently using R and training myself in it. However, in a lot of places, I have seen people using Python. What ...
69
votes
1answer
210k views

How to get correlation between two categorical variable and a categorical variable and continuous variable?

I am building a regression model and I need to calculate the below to check for correlations Correlation between 2 Multi level categorical variables Correlation between a Multi level categorical ...
49
votes
9answers
6k views

Is the R language suitable for Big Data

R has many libraries which are aimed at Data Analysis (e.g. JAGS, BUGS, ARULES etc..), and is mentioned in popular textbooks such as: J.Krusche, Doing Bayesian Data Analysis; B.Lantz, "Machine ...
46
votes
9answers
65k views

IDE alternatives for R programming (RStudio, IntelliJ IDEA, Eclipse, Visual Studio)

I use RStudio for R programming. I remember about solid IDE-s from other technology stacks, like Visual Studio or Eclipse. I have two questions: What other IDE-s than RStudio are used (please ...
35
votes
7answers
4k views

Organized processes to clean data

From my limited dabbling with data science using R, I realized that cleaning bad data is a very important part of preparing data for analysis. Are there any best practices or processes for cleaning ...
29
votes
3answers
39k views

Hypertuning XGBoost parameters

XGBoost have been doing a great job, when it comes to dealing with both categorical and continuous dependant variables. But, how do I select the optimized parameters for an XGBoost problem? This is ...
24
votes
9answers
20k views

Any Online R console?

I am looking for an online console for the language R. Like I write the code and the server should execute and provide me with the output. Similar to the website Datacamp.
24
votes
5answers
3k views

VM image for data science projects

As there are numerous tools available for data science tasks, and it's cumbersome to install everything and build up a perfect system. Is there a Linux/Mac OS image with Python, R and other open-...
18
votes
6answers
12k views

What do you use to generate a dashboard in R?

I need to generate periodic (daily, monthly) web analytics dashboard reports. They will be static and don't require interaction, so imagine a PDF file as the target output. The reports will mix tables ...
18
votes
3answers
12k views

is there any data tidying tool for python/pandas similar to R tidyr tool?

I'm working on a Kaggle challenge where some variables are represented by rows instead of columns (Telstra Network Disruption). I am currently searching for the equivalent of gather(), separate() and ...
17
votes
2answers
7k views

Recommending movies with additional features using collaborative filtering

I am trying to build a recommendation system using collaborative filtering. I have the usual [user, movie, rating] information. I would like to incorporate an ...
16
votes
3answers
46k views

How to predict probabilities in xgboost?

The below predict function is giving -ve values as well so it cannot be probabilities. ...
15
votes
4answers
7k views

R: machine learning on GPU

Are there any machine learning packages for R that can make use of the GPU to improve training speed (something like theano from the python world)? I see that there is a package called gputools which ...
15
votes
1answer
96k views

removing strings after a certain character in a given text

I have a dataset like the one below. I want to remove all characters after the character ©. How can I do that in R? ...
15
votes
0answers
882 views

Visualization of multiple Markov models

I am working on a project where we compare over 10 different Markov models, each representing a different treatment plan. Most often single models are visualized with a decision tree or transition ...
14
votes
4answers
12k views

Is pandas now faster than data.table?

https://github.com/Rdatatable/data.table/wiki/Benchmarks-%3A-Grouping The data.table benchmarks hasn't been updated since 2014. I heard somewhere that Pandas is ...
14
votes
5answers
10k views

Do modern R and/or Python libraries make SQL obsolete?

I work in an office where SQL Server is the backbone of everything we do, from data processing to cleaning to munging. My colleague specializes in writing complex functions and stored procedures to ...
12
votes
3answers
959 views

Predicting next medical condition from past conditions in claims data

I am currently working with a large set of health insurance claims data that includes some laboratory and pharmacy claims. The most consistent information in the data set, however, is made up of ...
11
votes
2answers
3k views

Is a 100% model accuracy on out-of-sample data overfitting?

I have just completed the machine learning for R course on cognitiveclass.ai and have begun experimenting with randomforests. I have made a model by using the "randomForest" library in R. The model ...
11
votes
4answers
18k views

How to avoid overfitting in random forest?

I want to avoid overfitting in random forest. In this regard, I intend to use mtry, nodesize, and maxnodes etc. Could you please help me choose values for these parameters? I am using R. Also, if ...
11
votes
3answers
1k views

What regression to use to calculate the result of election in a multiparty system?

I want to make a prediction for the result of the parliamentary elections. My output will be the % each party receives. There is more than 2 parties so logistic regression is not a viable option. I ...
11
votes
1answer
2k views

GPU Accelerated Data Processing for R in Windows

I'm currently taking a paper on Big Data which has us utilising R heavily for data analysis. I happen to have a GTX1070 in my pc for gaming reasons. Thus, I thought it would be really cool if I could ...
11
votes
1answer
517 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet appears to use the coordinate descent method to solve the ...
10
votes
5answers
27k views

LSTM or other RNN package for R

I saw some impressive result from LSTM models producing Shakespeare like texts. I was wondering if an LSTM package exists for R. I googled for it but only found packages for Python and Julia. (maybe ...
10
votes
4answers
1k views

Learning ordinal regression in R?

I'm working on a project and need resources to get me up to speed. The dataset is around 35000 observations on 30 or so variables. About half the variables are categorical with some having many ...
10
votes
1answer
9k views

Do you have to normalize data when building decision trees using R?

So, our data set this week has 14 attributes and each column has very different values. One column has values below 1 while another column has values that go from three to four whole digits. We ...
10
votes
4answers
4k views

Visualizing items frequently purchased together

I have a dataset in following structure inserted in a CSV file: Banana Water Rice Rice Water Bread Banana Juice Each row indicates a collection of items ...
10
votes
2answers
544 views

Software Testing for Data Science in R

I often use Nose, Tox or Unittest when testing my python code, specially when it has to be integrated with other modules or other pieces of code. However, now that I've found myself using R more than ...
10
votes
3answers
898 views

Data visualization for pattern analysis (language-independent, but R preferred)

I want to plot the bytes from a disk image in order to understand a pattern in them. This is mainly an academic task, since I'm almost sure this pattern was created by a disk testing program, but I'd ...
10
votes
2answers
2k views

ggvis vs. ggplot2+Shiny; which one to choose for interactive visualization?

There is a similar question here in CrossValidated, and I have read the answers. My question is a bit different. I don't want to merely visualize my data, and indeed what I want to visualize is not ...
10
votes
1answer
3k views

What is the difference in xgboost binary:logistic and reg:logistic

What is the difference in R in xgboost between binary:logistic and reg:logistic? Is it only in evaluation metric? If yes, how does RMSE on binary classification compare to error rate? Is the ...
10
votes
1answer
824 views

Libraries for (label propagation algorithms/frequent subgraph mining) for graphs in R

General description of the problem I have a graph where some vertices are labeled with a type with 3 or 4 possible values. For the other vertices, the type is unknown. My goal is to use the graph to ...
10
votes
1answer
3k views

Convergence in Hartigan-Wong k-means method and other algorithms

I have been trying to understand the different k-means clustering algorithms mainly that are implemented in the stats package of the ...
9
votes
3answers
1k views

What are R's memory constraints?

In reviewing “Applied Predictive Modeling" a reviewer states: One critique I have of statistical learning (SL) pedagogy is the absence of computation performance considerations in the evaluation ...
9
votes
3answers
2k views

R random forest on Amazon ec2 Error: cannot allocate vector of size 5.4 Gb

I am training random forest models in R using randomForest() with 1000 trees and data frames with about 20 predictors and 600K rows. On my laptop everything works ...
8
votes
1answer
4k views

Which of the 180 algorithms in R's caret package are feasible?

R's caret package works with 180 models. The author warns that some of the package may be intractably slow or less accurate than top-choice models. The author is not wrong about this. I've tried to ...
8
votes
5answers
9k views

Convolutional Neural Networks in R

I don't see a package for doing Convolutional Neural Networks in R. Has anyone implemented this kind of algorithm in R?
8
votes
3answers
3k views

How to find similarity between different factors in a dataset

Introduction Let's say I have a dataset of different observation of different people and I want to group people together to know which person is closest to the other one. I also want to have a ...
8
votes
1answer
7k views

R error using package tm (text-mining)

I am attempting to use the tm package to convert a vector of text strings to a corpus element. My code looks something like this Corpus(d1$Yes) where d1$Yes is a factor with 124 levels, each ...
8
votes
1answer
6k views

R - Interpreting neural networks plot

I know there are similar question on stats.SE, but I didn't find one that fulfills my request; please, before mark the question as a duplicate, ping me in the comment. I run a neural network based on ...
8
votes
2answers
746 views

Best way to store large data set using R from Twitter?

I am working on a project that aims to retrieve a large data-set (i.e., tweet data which is a couple of days old) from Twitter using the twitteR library on R. have difficulty storing tweets because ...
8
votes
1answer
2k views

R, keras: How to get output of a hidden layer?

I am using package Keras in R to do a neural network. How may I extract the output from a hidden layer? I found an example in python, but it is just I have no idea ...
8
votes
2answers
2k views

visualize a horizontal box plot in R

I have a dataset like this. The data has been collected through a questionnaire and I am going to do some exploratory data analysis. ...
8
votes
3answers
504 views

Are there any machine learning techniques to identify points on plots/ images?

I have data for each vehicle's lateral position over time and lane number as shown in these 3 plots in the image and sample data below. ...
8
votes
1answer
325 views

How far can one go with excel? [closed]

in my business we handle all analytics through Excel. This includes mostly scheduling, production planning and accounting operations. We currently are looking into adding a bit of predictive modelling ...
8
votes
2answers
8k views

Information Gain in R

I found packages being used to calculating "Information Gain" for selecting main attributes in C4.5 Decision Tree and I tried using them to calculating "Information Gain". But the results of ...
8
votes
4answers
691 views

Classifying Email in R

I'm working on a project in R where I have roughly 1200 emails from a company, most of which are labeled class$_{1}$ or class$_{2}$, which are the types of requests. Roughly 1000 emails are labeled ...
8
votes
1answer
77 views

Features reduction for the not correlated data set

I am working with classification problem on a training data set, which have 100 features. All the features in pairs haven't visible correlation. One can see it in the example pair plot for the some of ...
7
votes
2answers
5k views

Recommender system based on purchase history, not ratings

I'm exploring options for recommender systems optimized for the insurance industry, which would take into account i) product holdings ii) user characteristics (segment, age, affluence, etc.). I ...
7
votes
3answers
171 views

Identifying templates with parameters in text fragments

I have a data set with text fragments having a fixed structure that can contain parameters. Examples are: ...