Skip to main content
Share Your Experience: Take the 2024 Developer Survey
33 votes
Accepted

Is Python a viable language to do statistical analysis in?

Python is more "general purpose" while R has a clear(er) focus on statistics. However, most (if not all) things you can do in R can be done in Python as well. The difference is that you need ...
Peter's user avatar
  • 7,526
31 votes

Is pandas now faster than data.table?

Has anyone done any benchmarks? Yes, the 2014's benchmark in question has turned into foundation for db-benchmark project. Initial step was to reproduce 2014's benchmark on recent version of software,...
jangorecki's user avatar
29 votes
Accepted

Is a 100% model accuracy on out-of-sample data overfitting?

High validation scores like accuracy generally mean that you are not overfitting, however it should lead to caution and may indicate something went wrong. It could also mean that the problem is not ...
Jan van der Vegt's user avatar
25 votes
Accepted

IDE alternatives for R programming (RStudio, IntelliJ IDEA, Eclipse, Visual Studio)

RIDE - R-Brain IDE (RIDE) for R & Python, Other Data Science R IDEs, Other Data Science Python IDEs. Flexible layout. Multiple language support. Jupyter notebook - The Jupyter Notebook App is ...
karupakalas's user avatar
22 votes

How to predict probabilities in xgboost using R?

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.
ihadanny's user avatar
  • 1,357
18 votes
Accepted

Do modern R and/or Python libraries make SQL obsolete?

R and SQL are two completely different beasts. SQL is a language that you can use to query data that is stored in databases as you already experienced. The benefits of SQL versus R lays mostly in the ...
Stereo's user avatar
  • 1,413
17 votes

Is pandas now faster than data.table?

A colleague and I have conducted some preliminary studies on the performance differences between pandas and data.table. You can find the study (which was split into two parts) on our Blog (You can ...
Tobias Krabel's user avatar
14 votes
Accepted

Which of the 180 algorithms in R's caret package are feasible?

Benchmarking mlr (default) learners on OpenML The entire openml database of ML results. Test from RStudio suggests SVM. Mlmastery suggests LDA and Trial and Error. Do we Need Hundreds of ...
ran8's user avatar
  • 343
13 votes
Accepted

Cutting numbers into fixed buckets

You don't really need to implement an algorithm to achieve this. There are a few tools that will do this for you. You can get the data assigned to buckets for further processing using Pandas, or ...
n1k31t4's user avatar
  • 14.9k
13 votes

Is Python a viable language to do statistical analysis in?

Python being more widely used is an important consideration. This will especially become important when applying for a job. Also Python has as many if not more key statistical and ML/AI tools as R, ...
Donald S's user avatar
  • 1,959
12 votes

IDE alternatives for R programming (RStudio, IntelliJ IDEA, Eclipse, Visual Studio)

You may try using R with Jupyter notebook. It requires installation of jupyter R kernel, IRkernel which will allow you to open a new jupyter notebook with option to choose ...
Samir's user avatar
  • 296
12 votes
Accepted

Clustering for multiple variable

K-means Your data has $7$ dimensions so k-means is worth to try. See the PCA of your data and check if any cluster is visible there as K-means will have a tough time if clusters are not Gaussian. the ...
Kasra Manshaei's user avatar
10 votes
Accepted

Package that is similar to R's caret?

MLR is similar to caret, MLR offers a high-level interface to various statistical and machine learning packages. According to the package description: Interface to a large number of classification ...
krishna Prasad's user avatar
10 votes

Is there a R implementation of isolation forest for anomaly detection?

See the iforest package on Sourceforge or on R-Forge: This package implements an anomaly detection method that detects data-anomalies using binary trees. Using the ...
rcs's user avatar
  • 720
10 votes

What is Orange all about?

As you mentioned, Orange is a data mining software developed by the University of Ljubljana. It can be used for developing and testing machine learning models as well as conducting exploratory data ...
Ethan's user avatar
  • 1,633
9 votes

k-means in R, usage of nstart parameter?

nstart option attempts multiple initial configurations and reports on the best one. For example, adding nstart=25 will generate 25 initial random centroids and choose the best one for the algorithm. ...
FrlUn's user avatar
  • 121
9 votes
Accepted

Logistic regression on biased data

Background I'll start with some background to help you research the solution yourself and then will add some specifics. What you refer to as "biased data" is more commonly known as ...
AN6U5's user avatar
  • 6,818
9 votes

How to replace NA values with another value in factors in R?

You can use this function : forcats::fct_explicit_na library(forcats) fct_explicit_na(DF$col, na_level = "None") Usage It can be used within the mutate ...
eg-r's user avatar
  • 176
8 votes

Python vs R for machine learning

An issue all other answers fail to address is licensing. Most of the aforementioned wonderful R libraries are GPL (e.g. ggplot2, data.table). This prevents you from distributing your software in a ...
8 votes

LSTM or other RNN package for R

Have a look at the rnn package (full disclosure, I am the author). It implements a multilayer RNN, GRU, and LSTM directly in R, i.e. not an underlying C++ library, so you should also be able to read ...
Bastiaan Quast's user avatar
8 votes

Is there any data tidying tool for python/pandas similar to R tidyr tool?

I'd start with the melt() function in pandas. I wrote an article about it: https://www.ibm.com/developerworks/community/blogs/jfp/entry/Tidy_Data_In_Python?lang=en
JFP's user avatar
  • 81
8 votes

R, keras: How to get output of a hidden layer?

You can get the answer here. Here it is: ...
Perochkin's user avatar
  • 311
7 votes

LSTM or other RNN package for R

Keras is also now available for R. Here's an example of an LSTM with the R API.
captainpete's user avatar
7 votes

How to group by multiple columns in dataframe using R and do aggregate function

...
Cabana's user avatar
  • 171
7 votes
Accepted

Pros and Cons of Python and R for Data Science

Interaction - Random Facts Both are good stable languages with interesting complementary qualities. You can get much better packages in one and then stitch them with some data from the other. An ...
Abhishek's user avatar
  • 1,969
7 votes

Remove part of string in R

You may use gsub function > c <- "ce7382" > gsub("[a-zA-Z ]", "", c) [1] "7382" Feel free to add other characters ...
Marmite Bomber's user avatar
7 votes
Accepted

R vs. Python Decision Tree

Decision trees involve a lot of hyperparameters - min / max samples in each leaf/leaves size depth of tree criteria for splitting (gini/entropy) etc Now different packages may have different ...
Vivek Kalyanarangan's user avatar
7 votes

How do Data Scientists integrate Predictive Modeling with SQL?

There is no need to download data in CSV format, in most cases it is actually bad practice. Consider cases where data size > 1GB and is updated daily. This would add a considerable overhead and is not ...
mincorp's user avatar
  • 71
7 votes
Accepted

Reticulate vs Python

I actually tested this recently on a random forest fitting using two approaches: Using Jupyter notebooks to fit my model via python with data that had been tidied ...
Fnguyen's user avatar
  • 1,743
7 votes
Accepted

SMOTE for regression

I think SMOGN will work for your problem. The method is described in a paper titled: "SMOGN: a Pre-processing Approach for Imbalanced Regression". You can find it on arXiv. There is also a python ...
Mojtaba's user avatar
  • 86

Only top scored, non community-wiki answers of a minimum length are eligible