Questions tagged [r]
R is a free, open-source programming language and software environment for statistical computing, bioinformatics, and graphics.
1,488
questions
5
votes
5
answers
50k
views
Additive vs Multiplicative model in Time Series Data
The above time series plot is a daily closing stock index of a company. I want to know which model between additive and multiplicative best suits the above data. I know what the two models are, but i ...
- 231
5
votes
1
answer
28k
views
Clustering for multiple variable
There are total 50 students(john, Roy..) and used some action to do a job. My dataSet something like this
...
5
votes
8
answers
657
views
Skills that school doesn't teach you
I'll be graduating soon with a master's degree in electrical engineering. I have a lot of experience with the theoretical side of machine learning through courses and research, but all of my ...
- 51
5
votes
1
answer
3k
views
Package that is similar to R's caret?
Recently I stumbled upon a R package that works similar like caret, but I can't remember what it's name was and I can't find it. It seemed to be less well known, but at least as extensive. Someone ...
- 752
5
votes
2
answers
13k
views
Caret and rpart - does caret automatically prune rpart trees
Question relating to the caret package 'rpart' method.
Does the method='rpart' automatically prune the tree? If so, what rules does it follow? If not, how does one go about directing caret to do ...
- 153
5
votes
3
answers
3k
views
ICD-10 codes in Machine Learning
Can anyone provide specific techniques with using ICD-10 codes in Machine Learning? I have usually used a simply approach of creating multiple binary column representing ICD-10 codes… which can get ...
- 53
5
votes
1
answer
9k
views
Best or recommended R package for logit and probit regression
Could somebody please recommend a good R package for doing logit and probit regression? I have tried to find an answer by searching on Google but all the links I find go into lengthy explanations ...
- 53
5
votes
1
answer
672
views
How to manually calculate the variance of the least squares estimator in R
As stated in the title, how do you manually calculate the variance of the least squares estimator in R?
I know that the least estimates have the following formula:
$$\hat{\beta}=(X^TX)^{-1} X^T Y, $$
...
- 53
5
votes
2
answers
2k
views
Variable Importance Random Forest on R
I am currently using a random forest model for classification, however I am unsure how the feature selection technique "varImp" works on R. I understand the context of variable importance, however ...
- 103
5
votes
4
answers
1k
views
Determine highly correlated segments
Given a dataset that has a binary (0/1) dependent variable and a large collection of continuous and categorical independent variables, is there a process and ideally a R package that can find ...
- 151
5
votes
1
answer
50k
views
How to plot multiple columns with ggplot in R?
I do have a data frame with different categorical and numerical columns with the following schema:
Id | num_col_1 | num_col_2 | num_col_3 | cat_col_1 | cat_col_2
...
- 1,007
5
votes
3
answers
703
views
R has {drake} which makes it easy to make reproducible data pipelines. Does Python have a similar package?
See R's {drake}. It allows you to define a reproducible pipeline
...
- 630
5
votes
2
answers
275
views
Data visualization with extreme far away points
I want to show points across two groups. However, for both groups, there are some points which are far away from most of the other points within each group, shown below. Any suggestions for this ...
- 187
5
votes
1
answer
2k
views
How to subset rows from a data frame with comparison operators in R
I have a data frame (a csv file) with dimensions 100x6 and I need only the columns c("X1", "X2", "X4") and the rows in which the value of "X1" is greater than 30. So I did:
...
5
votes
3
answers
5k
views
R and Python, memory differences yielding performance differences
I think there are numerous posts regarding which one to use: R or Python. However, I'm curious about how their architecture differences yield differences in speed performance, not which one to use.
...
- 685
5
votes
3
answers
5k
views
Stochastic gradient descent in logistic regression
I am very new to machine learning and in my first project have stumbled across a lot of issues which I really want to get through.
I'm using logistic regression with R's ...
- 81
5
votes
1
answer
113
views
experimental design in R project
I want to know of any repositories that contain complete experimental design in R covering basic test and analyses?
I want to take a top-buttom approach to learn step by step through a real project ...
- 171
5
votes
2
answers
401
views
How do tech-companies employ Random Forest on large data sets?
The algorithm takes quite a long time to train on large data sets with a moderate number of parameters:
https://stats.stackexchange.com/questions/37370/random-forest-computing-time-in-r
https://...
- 338
5
votes
2
answers
8k
views
Overfitting XGBoost
I try to classify data from a dataset of 315 lines and 17 (real data) features (315x17). The target value is either "good" or "bad" (binary classification).
I used XGBoost to classify these data, ...
- 71
5
votes
2
answers
9k
views
Is there an R tutorial of using LSTM for multivariate time series forecasting?
There is a great blog post about how to use keras stateful LSTM in R to forecast sunspots. I applied it to financial ts data ...
- 151
5
votes
2
answers
4k
views
5
votes
3
answers
986
views
Classification technique for unsupervised data?
I have unsupervised data (i.e this data doesn't have any target variable through which I can learn it's prior behaviour) it is a mix of continuous and categorical data. Now I want to classify the test ...
- 184
5
votes
1
answer
2k
views
Financial Time Series data normalization
I'm using Keras in R to predict financial time series. It's easy to normalize price, simply compute returns or log returns, usually it's enough. I want to use Goldman Sachs Financial Conditions Index ...
- 85
5
votes
1
answer
2k
views
How to find the relationship between categorical variables?
I have a dataset where I have the Outcome of a case and the Country under review, as well as whether a judge (among a panel of ...
- 51
5
votes
1
answer
12k
views
What is the best way to deal with imbalanced data for XGBoost? [closed]
There are a lot of way to deal with class-imbalanced data like undersampling, oversampling, changing cost function etc.
https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-...
- 211
5
votes
2
answers
506
views
5
votes
2
answers
3k
views
Slow convergence with rpart
I'm building a decision tree in R using the rpart function, available in the library of the same name, but am experiencing some serious performance issues when ...
- 1,483
5
votes
1
answer
5k
views
Testing bimodality of data
I am trying to see if my data is multimodal (in fact, I am more interested in bimodality of the data). I performed dip test and it does evidence against unmodal data. However, I want to see, in ...
- 1,091
5
votes
1
answer
785
views
Clustering for high dimensional data
I am have a data set with 52 variables. Most of them have zeros, it resembles a sparse matrix. How can I cluster this kind of data and are there any special types of clustering? I am attaching pca ...
- 373
5
votes
1
answer
415
views
Sales Prediction for Fashion Retail Data
I am working on project related to prediction of sales quantity based on previous data.
Data consists of attributes like .MRP, Size , Discount,Color,Fabric type e.t.c
I am having both sales ...
- 137
5
votes
2
answers
3k
views
Constrained k-means algorithms in R (must-link constraints)
I currently face an unsupervised learning task that is to be approaches using clustering. More specifically, it is a segementation task and hence there is some prior knowledge about a) the number of ...
- 51
5
votes
4
answers
505
views
Categorical Clustering of Users Reading Habits
I have a data set with a set of users and a history of documents they have read, all the documents have metadata attributes (think topic, country, author) associated with them.
I want to cluster the ...
5
votes
1
answer
103
views
Finding aggregated information of data
I am new to data science. I have a dataset of around 200,000 records, having 5 columns. There is a field called, class. For each ...
- 335
5
votes
1
answer
313
views
Multivariate - Time series data pattern changes
New to R. In my example, my customers have restricted allocation of budget for Milk. I have more than 5 brands of milk in my store. Here my objective is how I know my customer is shifting from one ...
- 263
5
votes
2
answers
159
views
How to interpret two continous variables output using GAM?
I really need help with GAM. I have to find out whether association is linear or non-linear by using GAM. The predictor variable is temperature at lag0 and the output is cardiovascular admissions (...
- 51
5
votes
1
answer
651
views
k-Nearest Neighbours with time series data - how to obtain whole-time-period estimators
I have a large dataset for the activities performed by multiple staff in a factory over a long period of time - 01/01/2017 - present. The activities performed by the different staff are recorded at ...
- 151
5
votes
1
answer
17k
views
Plot of ACF & PACF
There are 96 observations of energy consumption per day from 01/05/2016 - 31/05/2017. I am trying an ARIMA model in R to be fitted to these time series observations. I have chosen the frequency of ...
- 429
4
votes
5
answers
145k
views
How to replace NA values with another value in factors in R? [closed]
I have a factor variable in my data frame with values where in the original CSV "NA" was intended to mean simply "None", not missing data. Hence I want replace every value in the given column with "...
- 8,587
4
votes
3
answers
796
views
machine learning on athlete performances to predict the time in a future race
Example Data
I have a dataset (in R as a data frame) of race results for athletes.
...
- 141
4
votes
4
answers
98
views
How do Data Scientists integrate Predictive Modeling with SQL?
I've heard a lot about the importance of knowing SQL but how do you guys at the larger firms integrate your software for modeling/analysis with the server that holds the data?
Do you simply access ...
- 141
4
votes
7
answers
145k
views
Remove part of string in R
I have a table in R. It just has two columns and many rows. Each element is a string that contains some characters and some numbers. I need number part of the element. How can I have number part?
For ...
- 41
4
votes
1
answer
17k
views
SMOTE for regression
I am looking into upsampling an imbalanced dataset for a regression problem (Numerical target variables) in python.
I attached paper and R package that implement SMOTE for regression, can anyone ...
- 735
4
votes
4
answers
477
views
Non-brute force approach to finding permissible English word anagrams
I am interested in proposed base R/Python solutions (i.e. without relying on external packages/libraries) to the following problem:
You have a large dictionary containing millions of valid English
...
- 1,216
4
votes
4
answers
2k
views
How to visualise multidimensional categorical data with additional time dimension
I am trying to visualize a dataset that registers the career of people in an organisation and their backgrounds before starting in that particular organisation. I 'd like to show how individuals are ...
- 141
4
votes
1
answer
770
views
Multiclass classification with Neural Networks
Let’s suppose I wanted to classify some input as one of three categories using a simple neural network.
The output of my network are three columns (one for each possible category I assume) with values ...
- 95
4
votes
1
answer
634
views
Importing Common Constants to R and Python
I use Python for data munging, R for data analysis, and I combine both by running the munging/analysis sequentially in a Makefile. My Python and R scripts depend on various constants. I typically ...
- 143
4
votes
2
answers
5k
views
How can we evaluate DBSCAN parameters?
yes, DBSCAN parameters, and in particular the parameter eps (size of the epsilon neighborhood).
In the documentation we have a "Look for the knee in the plot".
Fine, but it requires a visual analysis....
- 277
4
votes
1
answer
3k
views
difference between logistic regression and binary logistic regression
In xgboost R package, there are two objectives given with booster gbtree.
reg:logistic
...
- 451
4
votes
4
answers
3k
views
oversampling plus down sampling using smote not working on random forests
I am trying to solve a classification problem on a highly imbalanced data set. I am using SMOTE to over sample the minority samples and down sample the majority ones. After creating a balanced data ...
- 141
4
votes
1
answer
1k
views
Overfitting - how to detect it and reduce it?
I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
- 85